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REASONING  ABOUT  ARRAYS* 

John  C.  Reynolds 
Syracuse  University 


ABSTRACT 

A variety  of  concepts,  laws,  and  notations  are  presented  which 
facilitate  reasoning  about  arrays.  The  basic  concepts  include  intervals 
and  their  partitions,  functional  restriction,  images,  pointwise  extension 
of  relations,  ordering,  single-point  variation  of  functions,  various 
equivalence  relations  for  array  values,  and  concatenation.  The 
effectiveness  of  these  ideas  is  illustrated  by  informal  descriptions  of 
algorithms  for  binary  search  and  merging,  and  by  a short  formal  proof. 


* 
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1.  Introduction 

The  use  of  assertions  to  describe  programs  and  prove  their  correct- 

(123) 

ness  ’ ’ has  developed  to  the  point  where  the  necessary  assertions  are 
often  at  least  as  lengthy  and  difficult  to  comprehend  as  the  program 
which  they  describe.  A major  cause  is  the  use  of  languages  and  proof 
methods  - typicsiliy  the  first-order  predicate  calculus  - which  are  taken 
from  classical  logic  and  are  not  oriented  towards  programming. 

Perhaps  the  most  glaring  exeunple  of  these  difficulties  is  the  use  of 
arrays.  One  need  only  compare  the  assertions  needed  to  describe  a program 
such  as  n log  n exponentiation,  which  does  not  involve  arrays  or  other 
compound  data  structures,  with  the  assertions  for  a program  such  as  binary 
scorch,  which  is  intuitively  no  r.ore  complex,  but  uses  arrays.  In  the 
first  case,  the  assertions  are  clear  and  concise,  and  reasoning  about  them 
involves  only  the  familiar  laws  of  elementarj-  algebra.  But  when  array's 
are  introduced,  the  assertions  become  lengthy  and  filled  with  quantifiers, 
and  their  manipulation  seems  only  tenuously  connected  with  the  programmer's 
intuition . 

Superficially,  we  need  a better  notation  for  assertions  about  arrays. 
But  more  fundamentally,  we  need  concepts  and  laws  which  are  not  only 
correct  but  also  reflect  our  intuitive  mderstanding  of  arrays,  just  as 
the  concepts  of  addition  and  multiplication,  and  the  associative, 
commutative,  and  distributive  laws  reflect  our  intuitive  \mderstanding  of 
numbers.  Once  the  right  concepts  and  laws  have  been  found,  it  is 
comparatively  trivial  to  design  a notation  which  facilitates  their 
application. 

This  paper  presents  a variety  of  concepts , laws , and  notations  for 
reasoning  about  arrays  - some  borrowed  from  mathematics  and  others 
original  - which  we  believe  meet  the  above  criteria.  Their  utility  will 
be  demonstrated  both  by  informal  descriptions  of  program  behavior  and  by 
a short  formal  proof  of  program  correctness. 

For  a progrsunming  language,  we  will  use  Algol  60  with  the  following 
changes : 

(1)  while  statements. 



(2)  Round  rather  than  square  brackets  for  array  subscripts 

(which  emphasizes  the  view  that  array  values  are  functions). 


i 

S 

it 

!» 


I 
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(3)  Integer  expressions  of  the  form  lover  X and  upper  X, 
denoting  the  minimum  and  maximum  subscripts  of  a one-dimensional 
array  X. 

(Although  we  will  not  use  procedures  here,  it  should  be  noted  that  for 
(3)  to  be  fully  usef\il,  there  must  be  some  way  of  restricting  the 
interval  of  subscripts  of  an  actual  array  parameter.) 

V/e  have  purposely  stayed  close  to  Algol  to  avoid  inadvertently 
choosing  a programming  language  which  hid  the  defects  of  our  assertion 
language.  In  particular,  we  have  refrained  from  introducing  our 
notation  for  assertions  into  the  programming  language  itself  (except  for 
lower  and  upper,  which  were  irresistably  attractive).  Moving  in  this 
direction  seems  to  lead  to  a very  high  level  language,  closer  to  APL  than 
to  Algol,  which  is  beyond  the  scope  of  this  paper. 

On  the  other  hand,  even  the  choice  of  Algol  has  had  subtle  effects 
on  the  ensuing  development.  For  example,  switching  to  a programming 
language  with  the  novel  approach  to  arrays  described  in  Chapter  11  of 
Reference  h would  necessitate  minor  changes  to  m.any  concepts,  such  as 
abandoning  the  ijniqueness  of  the  array  value  with  an  empty  domain. 

To  an  even  greater  extent  than  is  indicated  by  the  explicit 


references,  this  work  is  built  upon  the  ideas  of  C.  A.  R.  Hoare. 


(5,6,7) 


Mention  should  also  be  made  of  distinct  but  related  work  on  arrays  by 
D.  C.  Cooper^®^  and  of  work  by  R.  Burstall^^^  which,  roughly  speaking, 
does  for  list  structures  what  we  are  trying  to  do  for  arrays. 


2.  Interval  and  Partition  Diagrams 

Before  considering  arrays  themselves,  we  introduce  some  diagrammatic 
expressions  for  making  assertions  about  subscripts.  Basically,  these 
expressions  are  a formalization  of  the  diagrams  which  are  traditionally 
drawn  by  programmers  when  describing  arrays. 

An  interval  is  a finite  consecutive  set  of  integers.  If  a and  b are 

expressions  denoting  integers,  then  a b , called  an  interval  diagram, 

is  an  exjiression  denoting  the  interval 

al  ~ H (i  I a < i 1 b}  . 

When  formulating  general  properties  of  interval  diagrams  (or  partition 
diagrams)  we  will  always  use  the  standard  form  a|  b| . But  when  using 
r.r.p.rams  to  make  assertions,  we  will  permit  more  flexibility. 
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Specifically,  at  either  end  of  an  interval  diagram,  |a  may  be  written 

instead  of  a-l| . Also,  0 may  be  written  as  an  abbreviation  for  | a e 

Thus 

|a  ^ =.{i  I a £ i £ b} 

la  = U I a £ i < b} 

a I It)  = {i  1 a < i < b} 

= {a}  . 

For  any  finite  set  S,  we  writers  to  denote  the  size,  or  nianber  of 
elements  in  S.  Thus 


a|  b I = ifb  - a^O  then  b - a else  0 


(2.1) 

This  use  of  a conditional  expression  to  describe  a fundamental  property 
of  a data  structure  is  a clear  symptom  of  a potential  source  of  error, 
i.e.,  the  possibility  that  a program  may  be  correct  for  one  case  of  the 
conditional  but  not  the  other.  To  emphasize  this  situation,  we  say  that 
the  interval  a I 


b|  is  regular  when  b - a ^ 0,  or  irregular  when  b - a 


< 0.  It  is  evident  that  a nonempty  interval  is  always  regular,  but  the 
empty  interval  can  be  either  regular  or  irregular.  (This  is  a slight 
abuse  of  language;  it  is  really  the  interval  diagram,  rather  than  the 
interval  itself,  which  is  regular  or  irregular.) 

From  interval  diagrams,  we  can  build  more  complex  entities  called 
partition  diagrams,  vrhich  describe  relationships  between  intervals. 

0’  ®1’ 

(a)  a- 


If  a„,  a,,  ...  , a are  expressions  denoting  integers,  then: 


a., 

a 

1 

n-1 

n 

(b)  a. 


n-1 


is  called  a partition  diagram. 

, i.e.  the  intervals  denoted  by 


diagrams  obtained  by  eliminating  all  but  an  adjacent  pair  of  lines, 
are  called  the  component  intervals  of  the  partition  diagram. 

(c)  a^  a^  , i.e.  the  interval  denoted  by  the  diagram  obtained  by 
eliminating  interior  lines,  is  called  the  total  interval  of  the 
partition  diagram. 

(d)  The  partition  diagram  is  a logical  expression  which  is  true  iff 
the  component  intervals  are  a partition  of  the  total  interval,  i.e., 
iff  the  component  intervals  are  disjoint  and  their  tinion  is  the  total 
interval. 
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'1 


pr 


As  with  interval  diafrrairc,  | a ray  he  written  in  place  of 

a-i I , and  a | _ in  place  of 


I a a 


Hi  us  for 


example , 


ljL_J 

0 

1 

c 

] is 

intervals  [ 

iL_J 

{i  I b < i ^ c}  are  disjoint  and  their  union  is  the  total  interval 
a ^ ={i  I a_<i£c}. 

The  nature  of  partitions  iirpli  ec  tliat  the  size  of  the  total  interval 

is  the  sum  of  the  sizes  of  the  compoi.ent  intervals: 

n 

^ implies  aj^  a._^|  T.  | [2.2) 


a. 

» • • d .. 

a 

0 

1 

n-1 

n 

As  shown 

in  the  Appendix 

, (2.2) 

of  partition  diagrams: 

a. 

« • • 3 _ 

a 

0 

1 

n-1 

n 

Iff  eith.er 


(2.3) 


'‘O  - ^1  - 


<a  ,<a  or  a->aT>...>a  ,>a 

- n-1  - n 0 - - - n-1  - n 


Mote  that  the  first  inequality  asserts  that  every'  component  interval  is 
repular,  v/hile  the  second  inequality  asserts  that  every  com.ponent  iui.erval 
is  empty. 

From  (2.3),  the  following  simple  cases  are  ot-/ious: 


b I is  always  true. 

^ iff  a |l7|  iff  a < b iff  | a is  nonempty. 


iff  a £ b £ c iff  b 


a 

10 

_.£l 

(2.M 

(2.5) 

(2.6) 


Kore  interestingly,  one  can  easily  derive  several  "diagrarmati cally 
natural"  rulef:  of  inference:  (liere  "line"  refers  to  any  vertical  line  in 
a diagra.m,  including  its  associated  expression.) 


Frasxire  From  a partition  diap,iTm  one  can  infer  any  diagram 
obtained  b.y  deleting  a line,  i.e., 

implies  , ....  . 

Adjacent  Dnp-lication  From  a partition  diagram  one  can  infer 
any  diagrer  obtained  by  replicating  a line  next  to  itself, 

i.e., 

implies 


(2.7) 


(2.8) 


Substitution  From  two  partition  diagrams  such  that  the 
end  lines  of  the  first  match  some  pair  of  adjacent  lines 
in  the  second,  one  can  infer  the  diagram  obtained  by 
substituting  the  first  diagram  for  the  adjacent  lines  in 
the  second: 


(2.9) 


...  b, 

k 

c 

& 

3 

1 

1^  implies 

:;:3 

• • « 1^. 

k 

c 

The  use  of  these  rules  is  illustrated  by  the  following  inferences  , 
which  will  be  pertinent  to  the  binary  search  example  given  later: 

(a)  For  any  integers  I and  u,  (2.>i)  and  (2.8)  show  that 

rr- 


holds . 


(b)  Suppose 
(2.9), 
implies 


1 

a b 

u 

I 

t 1 

T 

u 

J+1 


and  a £ j £ b.  Then  by  (2.6)  and 
holds.  In  turn,  by  (2.7),  this 
and 


u 


j-1 


3.  Functions  ns  Array  Values 


There  are  two  quite  different  concepts  of  an  array.  The  more 
traditional  view  is  that  an  array  of,  say,  real  numbers  is  a function 


from  subscripts  into  variables,  which  in  turn  possess  real  values.  The 

.(5,6) 


more  recent  view,  expounded  by  Hoare  ’ and  Dijkstra  , is  that  an 
array  of  real  numbers  is  a variable  whose  value  is  a function  from 
subscripts  into  real  numbers.  In  this  paper,  we  take  the  latter  view. 

The  effect  is  to  banish  the  possibility  of  "sharing"  or  "aliasing"  among 
array  elements,  which  would  greatly  complicate  the  problems  of  proving 
program  correctness. 

Specifically,  we  assume  that  an  array  declared  by  t array  X(a:b)  is 
a variable  whose  values  range  over  the  set  of  functions  from  the  interval 
into  the  set  t. 


For  any  function  X,  we  write  dom  X for  the  domain  of  X and,  when 
this  domain  is  an  interval,  lower  X and  upper  X for  the  integers  such 


dom  X = lower  X upper  X| . This  definition  of  lov>*er  and  upper  is 
Intentionally  incomplete  for  the  case  where  X is  the  unique  function, 
denoted  by  <>,  whose  domain  is  the  empty  set. 
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When  SC  dom  X,  we  write  X *1  S,  called  the  restriction  of  X to  S, 
to  denote  the  function  such  that 

WX  'I  S)  = S (3.1) 

(Vi  e S)  (X^  S)(i)  = X(i)  . (3.2) 

This  concept,  which  mirrors  the  informal  idea  of  (the  value  of)  a 
subarray  or  segment  of  an  array,  satisfies 

If  S'  C S Cdom  X then  (X^l  S)  1 S'  = S'  (3.3) 

X *1  {}  = <>  . (3.U) 

As  an  example , consider  the  program 

begin  int^er  i;  integer  array  Squares (-5:  5); 

integer  array  Pcssquare5(0;  5); 

irt^er  array  tIosquares(lli : 5); 

for  i :=  -5  wtil  5 do  Squares (i ) :=  i x i; 

for  i :=  0 until  5 do  Possqueres(i)  :=  i x i; 


At  the  program  point  indicated  by  the  ellipsis,  the  following  assertions 
will  hold; 

dom  S'^unr^s  = ~5  5 

lower  Squares  = -5 

upper  Squares  = 5 

(Vi  e -5  ^ ) Squares (i ) = i x i 

Possquares  = Squares  ']  |0  5 

Ilosquares  = Squares  ^ {}  = <> 

lower  liosquares  > upper  Nosquares 

The  expressions  lower  X and  upper  X occur  so  frequently  in  interval 
and  partition  diagrams  that  it  is  useful  to  adopt  conventions  for  eliding 
them  mambiguously . When  an  interval  or  partition  diagram  is  labelled 
with  a function  X,  lover  X may  be  omitted  from  the  right-  of  the  leftmost 
line  of  the  diagram,  and  upper  X mav  be  omitted  from  the  left  of  the 

Wn,  • - 

rightmost  line.  For  example,  X:  | a b stands  for 

lover  X I a b upper  X| . Moreover,  when  an  interval  diagram  is 

used  to  restrict  a f-unction  X,  the  label  can  also  be  elided.  For  example, 
X 'I  I a|  stands  for  X ^ [lower  X ~ a . 
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For  a function  X,  we  write  {x},  called  the  imapie  of  X,  to  denote 
the  set  {X(i)  | i e dom  X}  of  values  obtained  by  applying  X to  members  of 
its  domain.  (On  the  other  hand,  when  x is  not  a fvinction,  {x}  will 
denote  the  singleton  set  containing  x.)  Thus  for  example, 

{Possquares}  = {0,  1,  U,  9,  16,  25) 

{Possquares  (l  ~ 3 } = (1,  h,  9) 


(Squares  j |-2  2|}  = (0,  1,  U} 


It  is  easily  seen  that  images  possess  the  following  properties: 


SC  dom  X implies  {X  ^ S}  C (x)  (3.5) 

{<>}  = {}  (3.6) 

SOS'  = ^ X implies  {X}  = (X  *1  S)  U {X  *1  S')  (3.7) 

(X-I  [i]}  = {X(i)}  (3.8) 

Jt  (X)  < /Z  dom  X . (3.9) 


U.  Operations  on  Relations 

« 

Suppose  p is  a binary  relation  between  two  sets  U and  U' . Then  p , 

called  the  pointwise  extension  of  p , is  the  binary  relation  between  the 

« 

set  of  subsets  of  U and  the  set  of  subsets  of  U' , such  that  S p S'  holds 
if  and  only  if  x p x'  holds  for  all  x in  S and  x'  in  S'. 

VIhen  U and  U'  are  both  the  set  of  integers,  p covLLd  be  any  of  the 
relational  operators  of  Algol.  For  example,  (2,  3)  ^ (3,  and 

(2,  3)  ¥ {^,  5}  are  both  true,  while  (2,  3)  < (3,  ^),  (2,  3)  = (2,  3), 

and  (2,  3}  (2,  3)  are  all  false.  The  last  two  examples  demonstrate 

that  ^ is  not  the  negation  of  = (and  thereby  show  the  importance  of 
making  * explicit). 

The  pointwise  extension  of  any  relation  satisfies  the  following  laws: 


S p S'  & T £ S implies  T p S' 

(^.la) 

S p S'  & T'  C S'  implies  S p T' 

(U.lb) 

{}  p*  S' 

(U.2a) 

S p*  {} 

(U.2b) 

(S  U T)  p*  S'  iff  S p*  S'  & T p*  S' 

(U.3a) 

S p*  (S'O  T')  iff  S p*  S'  & S p*  T' 

(«4.3b) 

(x)  p (x' } iff  X p x'  . 

(U.I4) 
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Occasionally,  one  needs  the  pointwise  extension  of  a relation  with 
regard  to  only  a single  argtment.  The  simplest  way  of  encompassing  this 

case  is  to  regard  x p S'  as  an  abbreviation  for  {x}  p S'  and  S p x' 

* 

as  an  abbreviation  for  S p lx'). 

Another  concept  involving  relations,  somewhat  more  specialized  than 

pointwise  extension,  is  ordering.  The  usual  idea  of  an  ordered  array  can 

be  generalized  to  an  arbitrary  relation  in  a way  which  unifies  several 

important  cases.  Let  X be  a function  whose  domain  is  a set  of  integers, 

and  let  p be  a binary  relation  appropriate  to  the  type  of  result  of  X. 

Then  X is  ordered  with  regard  to  p,  written  ord  X,  if  and  only  if,  for 

p 

all  i £ind  j in  the  domain  of  X,  i < j implies  X(i)  p X(j). 

The  following  "orderings"  appear  as  specific  cases: 


ord 

X: 

increasing  order 

ord^ 

X: 

strict  increasing  order 

ord 

X: 

decreasing  order 

ord 

X: 

strict  decreasing  order 

ord_ 

X: 

all  elements  equal 

ord^ 

X: 

all  elements  distinct 

Moreover,  the  generalization  satisfies  the  following  essential  laws  of 
ordering: 


ord  X & S C dom  X implies  ord  (Xi  S) 
v/W.  P — P 

IL  dom  X < 1 implies  ord  X 

■ - ~ p 

If  S U T = X & S <*  T then 

(ord  X iff  (ord  (xl  S)&ord  (X  1 T)  & {X  1 S)  p*  {X  1 T})) 

S.-~P  P ‘ V — p 


(4.5) 

(4.6) 

(»).T) 


An  important  special  case  of  (4.7)  is  obtained  by  taking  S and  T to  be 
two  components  of  a partition: 


If  X: 


I H -J 


then 


(ord  X iff  (ord  (X  '\ 
. — . p ' — P 


r k|  ) & onip(x1  k|  I ) (4.8) 

& ix"!  I iH)  P*  IX'l  k|  |}))  . 
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Binary  Search 


We  have  now  introduced  enough  of  our  notation  to  demonstrate  its 
use  in  describing  - precisely  yet  intelligibly  - why  a program  works. 

As  an  example,  we  describe  an  algorithm  for  binary  search. 

Given  an  ordered  array  X and  a test  value  y,  the  program  should 
set  the  boolean  variable  found  to  indicate  whether  any  element  of  X is 
equal  to  y.  If  fotind  is  true,  then  the  integer  variable  j should  be  set 
to  the  subscript  of  X such  that  X(j)  = y.  More  precisely,  if  ord^  X, 
then  executing  the  program  should  achieve  the  goal  ~ 


if  fotind  then  X: 


& X(j)  = y else  {X}  y . 


Throughout  program  execution,  found  will  only  be  set  to  true  if 

X:  I j I & X(j)  = y is  achieved.  On  the  other  hand,  when  found  is 

false,  it  will  not  be  known  that  y occurs  nowhere  in  X,  but  only  that  it 
does  not  occur  in  either  of  two  segments  at  the  left  and  right  ends  of  X. 
If  we  use  the  local  variables  a and  b to  delineate  these  segments,  we  have 
the  invariant : 


if  found  then  X: 


else  X: 


& X(j)  = y 

& {X  ( 1 la  U bl  ) )}  y . 


On  the  one  hand,  this  invariant  can  be  achieved  initially  by  setting 
found  to  false  and  making  the  end  segments  of  X empty.  On  the  other  hand, 
the  invariant  implies  the  goal  of  the  program  if  either  foimd  is  true  or 

a > b,  since  the  latter  condition  implies  that  a ^ is  empty  and  thus, 

from  the  partition  diagrairi,  □ a U b I I = dom  X.  Thus  oiir  program 


has  the  form: 


begin  integer  a,  b; 

a :=  lower  X;  b :=•  upper  X;  found  :=  false; 
while"! ( found  or  a > b)  do  ... 


V/hen  execution  of  the  body  of  the  while  statement  begins,  both  the 
invariant  and  the  while  test  will  be  true.  Since  a <^b,  we  can  perform  an 
operation  "Pick  j"  (whose  details  will  be  considered  later)  which  sets  j 
to  some  integer  in  _a ^ . At  this  stage,  we  will  have 

X:  I [a  I j I b|  I & {X'l  (I  |aU  b|  | )}  ^ y , 
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and  we  can  compare  X(j)  with  y.  There  are  three  cases: 


(1)  If  X(j)  = y,  the  invariant  will  he  preserved  if  fomid  is 
set  to  true. 

(2)  If  X(j)  < y,  then  ord^  X insvires  that  {Xl  ] J] } < y, 

so  that  {X^  (I  j^l  Ob|  ' | ))  y.  This  permits  us  to  set  a 

to  j + 1. 

(3)  If  X(j)  > y,  then  a similar  argument  justifies  setting  b 
to  j - 1. 


Ihvis  our  program  is: 


begin  integer  a,  b; 


while  "K found  or  a > b)  do 
begin 
"Pi^"  j"; 

if  X(j)  = y then  found  :=  true  else 

if  X(j)  < y then  a :=  j +1  else  b 

end 

end 


j - 1 


Termination  is  guaranteed  by  the  fact  that  each  iteration  either  sets 
found  to  true,  which  immediately  stops  further  iterations,  or  else  decreases 

, whose  emptiness  will  cause  termination.  The  absence 


the  size  of  | a b 


of  subscript  errors  is  guaranteed  since  X:  | | j | 

points  where  X(j)  is  evaluated. 


holds  at  the  program 


It  should  be  noticed  that  this  description  of  binary  search  does  not 

, and  therefore 


exclude  the  possibility  that  lov;er  X upper  X 


might  be  irregular.  The  heart  of  the  matter  is  the  reasoning  about 
partition  diagrams,  which  was  formalized  at  the  end  of  Section  2.  One  of 
the  virtues  of  this  kind  of  reasoning  is  that  it  includes  the  irregular 
case  without  any  special  case  analysis. 

To  complete  our  program,  we  must  digress  from  the  topic  of  arrays  to 
specify  "Pick  j".  In  this  case,  the  problem  is  not  to  find  a correct 
realization  - either  j :=  a or  j :=  b would  be  correct  - but  to  find  an 
efficient  one.  The  need  to  shrink  a b]  as  much  as  possible  suggests 
choosing  j at  or  near  the  midpoint  of  a b , i.e.,j  :={a+b)  ^-2. 
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However,/ we  must  be  sure  that,  if  a £b,  then  j :«=  (a  + b)  t 2 will 
achieve  a£  j 1.^,  despite  the  fact  that  integer  division  involves 
roxuiding  (and  that  the  details  of  this  rounding  might  vary  for  different 
machines,  especially  when  a + b is  negative).  Fortxmately , it  is  enough 
to  know  that  division  by  two  is  a monotonic  function  which  is  exact  for 
even  numbers.  For  a b implies  a + a£a+b£b  + b,  so  that 
monotonicity  gives  (a  + a)  t 2 ^ (a  + b)  v 2 £ (b  + b)  i 2,  and  exactness 
for  even  nvimbers  gives  a£(a+b)T2£b.  (S.  Winograd  has  pointed  out 

that  j :=(a+b)T2is  mnecessarily  prone  to  overflow,  in  comparison 
with,  for  example,  j :=  a + (b  - a)  •s-  2.  We  leave  it  to  the  reader  to 
show  that  the  correctness  of  his  improvement  can  still  be  proved  with  a 
monctonicity  argument.) 


6.  Array  Assignment 


We  must  now  move  beyond  programs  such  as  binary  search  which  merely 
use  arrays,  to  consider  pi'ograms  which  change  arrays.  In  programming 
languages  at  the  level  of  Algol,  the  fundamental  agent  of  change  is  an 
assignment  statement  which  alters  a single  array  element,  e.g.,  X(i)  :=  e. 

Hoare^^*^^  has  shown  that,  to  deal  with  this  statem.ent  from  the 
viewpoint  that  an  array  is  a function-valued  variable,  we  must  regard  it 
as  an  abbreviation  for  the  assignment  X :=  |^X  | i | e],  where  fx  | i | ej 
denotes  the  function  which  is  similar  to  X except  that  it  maps  i into  e. 
More  formally,  [[x  | i | e7  is  defined  when  i e dora  X,  in  which  case  it  is 
the  f\inction  satisiying 


dom  Lx  I i I e]  = dom  X (6.1) 

Qx  1 i I ej  (i)  = e (6.2) 

[x  I i I ej  (j)  = X(j)  when  j V i , (6.3) 

and,  as  an  immediate  consequence  of  (6.3), 

[x  I i I 01  S = Xl  s when  S C dom  X and  i ^ S . (6.U) 


Once  X(i)  :=  e is  seen  as  an  abbreviation  for  X :=  Tx  I i I ej , the 

(2) 

usual  axiom  of  assignment  ; 


(where  Pl 

' X e 

an  axiom  of  array  assignment 


<«-5> 

denotes  the  result  of  substituting  e for  x in  P)  extends  to 


{x  :=  e}  P 

esuH 

(6). 
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{X(i)  :=  e}  P . 


(6.6) 


'■'x-.Ex  I i I e] 

(Because  of  (6.1),  when  this  axiom  is  used,  the  substitution  X 
1 X I i I e^need  not  be  applied  to  occurrences  of  X in  dom  X,  X:,  lower  X, 

Vw  V/-w~ 

or  upper  X. ) 

7.  Equivalence  Relations  for  Arrays 

For  many  profjronis  which  alter  arraj''s , such  as  sorting  programs,  a 
ful]  specification  will  stipulate  both  that  the  final  value  of  the  array 
will  possess  some  property,  such  as  being  ordered,  and  that  the  final 
value  will  be  related  to  the  initial  value  in  some  way,  such  as  being  a 
rearrangement.  Often  ~ even  when  the  situation  is  intuitively  obvious  - 
a formidable  technical  apparatus  is  needed  to  foimulate  and  prove  the 
latter  kind  of  specification. 

To  deed  with  these  i)roblems  it  is  useful  to  introduce  several 
equivalence  relations  for  array  values.  Suppose  X and  Y are  both 
functions  whose  domains  are  sets  of  integers.  Then: 

(a)  Vfe  write  X “vv-  Y , and  say  that  X is  a redistribution  of  Y 
iff  {X}  = {Y}. 

(b)  V.’e  write  X a-  Y,  and  say  that  X is  a rearrangement  of  Y 
. iff  there  is  a bijection  B (sometimes  called  a one-to-one 

correspondence  or  a permutation)  from  dom  X to  dom.  Y such 
that  (vi  e dom  X)  Y(B(i))  = X(i). 

(c)  Ve  write  X = Y,  and  ssy  tha.t  X is  a shift  of  Y iff  there 
is  a bijection  as  in  (b)  with  the  special  form  E(i)  = i + s 
for  some  integer  s . 

This  defines  an  increasingly  stringent  sequence  of  equivalence  relations. 


Thus,  where  p is  ocv,  -v,  or  = : 

Transitivity  X p Y & Y p Z iniplies  X p Z (7.1) 

Symmetry  X p Y implies  Y p X (7.2) 

Reflexivity  X p X (7.3) 

X “ Y implies  X Y (7.*+) 

X 'v-  Y imqilies  X vv  Y . (7.5) 
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Finally,  we  have  three  more  specific  laws.  Exchanging  a pair  of 
elements  produces  a rearrangement: 

(Vi,  j e ^ X)CCx  I i I X(j)]  [ j ( X(i)J  X , (7.6) 

two  one-element  arrays  with  equal  values  are  shifts  of  one  another: 

Q]  = dom  X & Q]  = dom  Y & X{i)  = y(j)  implies  X «=  Y , (7-7) 

and  a shift  of  an  ordered  array  is  ordered: 

X « Y 8s  ordp  X implies  ord^  Y . (7.8) 

(3) 

As  Hoare  has  pointed  out,  for  any  program  which  only  alters  an 
array  by  performing  exchanges,  (7.1),  (7.3),  and  (7.6)  are  sufficient  to 
show  that  the  final  array  value  is  a rearrangement  of  the  initial  value. 
However,  to  deal  with  programs  which  move  information  from  one  array  to 
another,  we  u ’St  also  consider  the  concatenation  of  array  values. 


8.  Concatenation 

Let  X and  Y be  functions  whose  domains  are  intervals  with  sizes  m and 
n respectively.  Then  X'^Y,  called  the  concatenation  of  X and  Y is  the 
wique  function  such  that 


dom  (X  ^ Y) 


1 m+n 


(X*^  Y)  ']  \l  in 


X 


(X'^Y)'^  I m+l  m+n 


Y . 


Ihe  choice  of  one  as  a lower  bound  is  arbitrary,  since  we  will  always 
regard  shifts  of  concatenated  array  values  as  equivalent. 

Let  <>  denote  the  unique  function  whose  domain  is  enqpty.  Then 
concatenation  satisfies  the  following  laws: 


X » X 

(8.1) 

X “ X 

(8.2) 

(X^Y)'^Z  « X^(Y^Z) 

(8.3) 

X *=  X'  & Y - Y'  in^jlies  X Y « X*'^  Y' 

(8.U) 

X'^  Y -v  Y X 

(8.5) 

X a.  X'  & Y a,  Y'  implies  X Y a,  X'  Y' 

(8.6) 
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I 

I 


> 


I 


I 

I 


X: 


implies  X « (X'1  | a|  )'^  (X  'I  al  I ) 


(8.7) 


{X  = {X}  U {Y}  (9.8) 

ord  (X^Y)  iff  ord^  X & ord„  Y & {X}  p*  (Y)  . (8.9) 

vv-»  P W-  P P 

The  first  four  laws  show  that  array  values  form  a monoid  under 
concatenation,  provided  that  shift  eq\ii valence  is  used  in  place  of  true 
equality.  The  next  two  laws  show  that  this  monoid  becomes  commutative 
when  the  less  stringent  equivalence  of  rearrangement  is  used. 

(Technically,  one  can  make  these  statements  precise  by  working  with  the 
quotient  of  the  set  of  array  values  iinder  the  equivalence  relations 
“ or  -v . ) 

The  last  three  laws  establish  the  basic  connections  between 
concatenation  and  partitions,  images,  and  ordering.  In  particular,  (8.9) 
is  a consequence  of  (U.8)  and  (7.8). 

In  fact  (8,3)  actually  remains  true  when  = is  changed,  to  =.  But  the 
stronger  relationship  is  irrelevant,  since  we  sho\ild  never  be  interested 
in  true  equality  for  concatenated  array  values. 


9.  Verging 

As  a second  example  of  program  description,  we  consider  the  problem 
of  merging:  Given  two  ordered  arrays  X and  Y,  set  Z to  an  ordered 
rearrangem.ent  of  the  concatenation  of  X and  Y.  We  assum.e  that  Z is  just 
the  right  size  to  hold  the  res\ilt.  Thus  if 

ord^  X & ord  Y & $ dom.  Z = £f  dom  X + /^'  dom  Y , 
then  executing  the  program  should  achieve  the  goal 

ord^  Z & Z -v.  X'^Y  . 

During  execution,  each  array  will  be  partitioned  into  a processed 
part  on  the  left  and  an  unprocessed  part  on  the  right,  the  processed  part 
of  Z will  be  an  ordered  rearrangement  of  the  concatenation  of  the 
processed  parts  of  X and  Y,  the  luiprocessed  part  of  Z will  be  the  right 
size  to  hold  the  unprocessed  parts  of  X and  Y,  and  all  processed  elements 
in  Z will  be  smaller  or  equal  to  all  unprocessed  elements  in  X or  Y.  (The 
last  condition  is  needed  to  insxirc  that  the  unprocessed  elements  can  be 
moved  into  Z without  rearranging  the  already  processed  elements.)  Thus  we 
have  the  invariant: 
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I 


S X:  1 Ikx  1 & Y:  1 ky 

n & Z:  k^  f 

(a) 

*e  ord^  Z 1 ]kz 

(b) 

&Z1  I Ikz'v-Xl  1 |kx^ 

Yl  nzuky 

(c) 

&arZ:  fkz  1 = 32  X:  “1 

+ ZZY:  fky 

(d) 

& {Z  1 I Ikz}  < {X1  Hoi 

1}  U{y1  fky  } . 

(e) 

This  invariant  can  be  achieved  initially  by  making  the  processed 
parts  all  empty,  and  it  will  in^jly  the  goal  of  the  program  when  the 
unprocessed  parts  are  all  empty,  which  - by  (d)  - will  occur  when  the 
unprocessed  part  of  Z is  empty.  Thus  we  can  use  a program  of  the  form: 


begin  integer  kx,  ky,  kz; 

kx  :=  lower  X;  ky  :=  lower  Y;  kz  :=  lower  Z; 
while  kz  £ upper  Z do  "Copy  One  Element" 
end 


In  '.'Copy  One  Element",  a single  element  will  be  moved  from  the 
unprocessed  part  of  X or  Y into  the  processed  part  of  Z.  To  preserve 
cpndition  (e)  the  element  to  be  moved  must  be  the  smallest  member  of 
{X'l  |kx  1)0  {Y'I  I ky  1 } . Since  both  X and  Y are  ordered,  this  v;ill 
be  the  smaller  of  the  leftmost  xmprocessed  elements,  X(kx)  or  Y(ky), 
providing  both  unprocessed  parts  are  nonempty.  However,  if  only  one 
unprocessed  part  is  nonempty,  its  leftmost  element  will  be  the  element 
to  be  moved.  Thus : 


"Copy  One  Element"  = 

if  (if  kx  > upper  X thro  false  else 
if  ky  > upper  Y then  true  else 
X(kx)<Y(ky)) 

then  "Copy  X"  else  "Copy  Y"  , 
where,  prior  to  executing  "Copy  X" , 

IX  = Z:  Ikzl  1 & X:  |kx| 


& X{kx)  <.  (X'l  Ikx  I } U {Y*1  |ky  ]} 


(f) 

(g) 


will  hold  as  well  as  the  invariant  I. 
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Thus  (e)  will  he  preserved  if  "Copy  X"  moves  X(kx)  out  of  the 
lonprocessed  part  of  X and  into  the  processed  part  of  Z.  Moreover,  (e) 
insures  that  X(kx)  will  he  larper  or  equal  to  the  elements  which  have 
previously  been  moved  into  Z.  Thus  the  ordering  (b)  will  be  preserved 
if  X(kx)  is  placed  at  the  right  of  the  processed  part  of  Z.  Therefore; 

"Copy  X"  = 

begin  Z(kz)  :=  X{kx) ; kx  :=  kx  + 1;  kz  ;=  kz  + 1 end  , 
and  by  a similar  argvunent 

"Copy  Y"  H 

begin  Z(kz)  :=  Y(ky);  ky  :=  ky  + 1;  kz  :=  kz  + 1 end  . 

Formally,  in  the  notation  of  Reference  2,  "Copy  X"  must  meet  the 
specification 

I & IX  {"Copy  X"}  I . 


To  exemplify  the  application  of  the  various  laws  we  have  stated,  we  give 
a formal  proof  of  this  specification.  The  assignment  axioms  (6.5)  and 
(6.6)  imply  I'  ("Copy  X")  I,  where 


I' 


^^kz  kz+1  ^kx  kx+1  ^ Z (^Z  | kz  | X(kx)J 


= X:  [ 

k^l  1 & Y:  1 ky  1 & 

Z:  I kz]  j 

(a') 

& ord 

< 

fz  1 kz  1 X(kx)l1  I 

(b') 

&lz  1 

kz  I X(kx)  \ '\  I kz|  X n ]] 

kx!  YH  r Iky 

(C) 

& 2 Z: 

kz]  'I  = TL'k'.  kxf  I + iT  Y : 

( — I 

>1 

(dO 

& (tz 

I kz  I X(kx)l  n I kzl  } < {X 

'\  kxj  '|}U  {Y^  iky  ~ 

(Here  we  have  written  kx]  instead  of  |kx+l  and  kz|  instead  of  |kz+l.) 

'fhus  we  m.ust  show  that  I & IX  implies  I',  i.e.,  that  lines  (a)  through  (g) 
imply  (a')  through  (e')» 

By  the  rule  (2.9)  of  substitution,  (a)  and  (f)  inply 


X; 


Y: 


& Z: 


kz 


(h) 


which,  by  the  rule  (?.Y)  of  erasure,  implies  (a')  as  well  as  various 
partition  diagrams  used  in  the  sequel.  In  particular,  by  (2.2)  and  (2.1), 
X:  |kx|  } implies  JT  X:  [kx  I = /tX:  kxl  | + 1,  and  Z: 

= pL  Z:  kz 


kz 


implies  p Z: 


kz 


J +1,  so  that  (d)  implies  (d’). 
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Next , we  have 


I^Z  I kz  1 X(Kx)J'l  ] k-Z] 

= [;z  I kz  I X(kx)]1  I Ikz'^rz  I kz  I X(kx)]1 


n^).  (6.7) 


= Z'l  I [kz  [Z  I kz  I X(kx)J^ 

. Z1  □kz'^xl  B 
(X*!  I |kx^Yl  I |ky)^  X1 
(X1  Elllkx'^xl  (kil)"'Yl  I ^ky 
=:  X"l  I k^  ^Y']  CZlky 
which  establishes  (c'),  and  also 


(6.i*} 


(7. 7), (8. U), (6. 2) 
(c),(8.6) 

(8. 3), (8. 5), (8. 6), (7. »0 
(X:  Ikxl).  (8.7) 


[z  I kz  I X(kx)]'^  I kz|  - I |kz  X']  (kx|  . 


{[Z  I kz  I X(kx)3l  ! kzl  } 

= (Z1  I |kz^  X^  |k^  } 

= {z'l  I ikz)  u {xi  n^i) 

= {Z1  I |hz)  U {X(kx)} 

1 {X  'I  |kx  I } U {Y  'I  Iky  | } 


(i), (?.»*), (7. 5) 
(8.8) 

(3.8) 

(e),(g),(l+.3a) 


= {X^ 


X']  kxQ^}  U {Y1  Iky  |} 


(X: 

(8.8) 


).  (8.7) 


= {X^  U {X1  kx|  [}  U {y1  |ky  I)  (8.8) 

so  that  (U.la)  and  (U.lb)  give  (e')  and 

(Z*!  I fkz)  £ ex'!  |kx| } . (j! 

Finally,  (3.1),  (2.1),  and'(H.6)  imply  ord^  X"^  |j^  , which  with  (b), 
(j),  and  (8.9)  in^Jlies  ord^(Z'l  | Tkz‘*^X'1  |^  ) , which  with  (i)  and 

(7.8)  implies  (b'). 
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1-''.  M'jiti  t^ir:">imional  Arrays 


the  cr.ncepts  we  have  presented  were  developed  and  tested 
Ir  rhe  cri.l-ci:-;.  of  one-dimensional  arrays,  most  of  them  extend  to  the 
Tii r 1 limonsional  case.  The  major  additional  concept  which  is  needed  is 
the  Cartes i an  product : 


Sn  ^ 


& i e S } . 
n n 


i > I i e S,  & 
n ' 1 

A Cartesian  product  of  intervals  is  called  a hi ock . Ihe  values  of  the 

, a : b ) are  functions  whose 
* n n 


arrry  declared  ty  t array  X(a,  : b , . 

- i.  J. 


domain  is  the  block 

St 

b. 

X ...  X 

a 

b 1 

1 

1 

U}- 

_Jll 

It  is  evident  that  the  values  of  subai'rays  of  X such  as  rows  and 
eo]>:r;:G  are  restrictions  of  X to  certain  blocks.  For  example,  the 
folJowir.p;  asserts  that<'’,  j>is  a saddle  point  of  the  two-dimensional 


n rrey  >' : 

fxl  (|T]  X □□)}  <*  X(i,  j) 

& X{i,  j)  <*  {x1  (I  I X [3])}  • 


31.  Ccnclusion 

The  contents  of  this  paper  is  only  a sm.all  beginning.  It  is  largely 
limited  to  one-dimensional  integer-subscripted  arrays,  end  even  within  this 
domain  further  study  is  certain  to  produce  significant  extensions  and 
changes.  But  we  have  gone  far  enough  to  demonstrate  the  value  of  the 
underlying  approach:  Ve  have  formulated  concepts,  lav’s,  end  notations 
which  are  powerfvQ  tools  for  the  precise  yet  intelligible  description  of 
a sigr.ificant  aspect  of  programming. 

Hopefully,  this  work  suggests  guidelines  for  further  progress:  One 
should  focus  upon  particular  mechanisms  such  as  arrays  rather  than 
generalities  which  pertain  to  all  computation.  Concepts  and  laws  are  more 
fur.dr rental  than  notation  per  se , and  should  reflect  intuitive 
> rr.tar.dinr.  ”or.t  important,  the  crucial  test  is  the  ability  to 
d'--r.cribf-  r'-nl  program.s  in  a way  which  is  not  only  precise  but  also 
intelligible  to  the  human  reader. 
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APPENDIX 


Proof  of  Proposition  (g.3) 

We  leave  it  to  the  reader  to  verify  that  either  aQ 


o**  — °'l  — ■ 


^ a implies  a 


a-. 

...  a 

1 

< a 
— n 


The  following  proof 


of  the  converse  was  found  by  F.  L.  Morns. 

Suppose  SqI  aTl  . . . a_^ | . From  (2.2)  we  have 


a. 

...  a 

1 

n 

^ “ol  ^ = J,  « »i-ll  M 

1=1  ' ' 


(a) 


where 


"ff  a|  b|  = if  b - a ^ 0 then  b - a else  0 

is  always  nonnegative  and  is  zero  iff  af  b|  is  empty.  For  arbitrary  a^'s 
simple  cancellation  gives 

n 

a - a-  = y a.  - a.  , . 
n 0 1 1-1 

1=1 

Ihen  subtraction  of  (a)  from  both  sides  gives 


1=1 


(b) 


where 


f(a,  b)  = b - a - a|  b|  = ifb-a^0  then  0 else  b - a 


is  always  nonpositive  and  is  zero  iff  a(  b is  regular. 

[must  be  either  empty  or  regular  (or  both). 


The  interval  a. 


Suppose  it  is  empty.  Then  (a)  asserts  that  a sum.  of  nonnegative  terms 
is  zero,  which  implies  that  each  term  is  zero.  Thus  for  each  i, 
is  empty,  and  • 

I is  regular.  Then  (b)  asserts 


^i-1 


On  the  other  hand,  suppose  a^ 


that  a sum  of  nonpositive  terms  is  zero,  which  implies  that  each  term  is 

I is  regular,  and  £ a^. 


zero. 


Thus  for  each  i , a^_^ 
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EVALUATION  MODELS  FOR  INDEX  SEQUENTIAL  FILES 
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Abstract 

The  purpose  of  this  report  is  to  develop  and  illustrate  the  use  of 
probabilistic  models  for  times  spent  in  various  operations  on  index 
sequential  and  other  simpler  files.  Various  criteria  for  evaluating 
the  performance  of  a file  organization  are  described  and  the  problem 
of  tackling  variability  in  performance  is  discussed  and  a detailed 
description  of  an  index  sequential  file  and  operations  on  the  file  is  given. 
Models  for  read,  add,  update  and  delete  operations  on  an  index  sequential 
file  are  developed  and  an  explanation  of  the  use  of  basic  models  for 
special  cases  is  presented.  Based  on  these  models,  expressions  for 
processing  times  for  sequential  and  direct  files  are  developed.  Methods 
for  numerical  transformations  of  random  variables  needed  for  these  models 
are  also  discussed. 


i 
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1.  PERFORMANCE  OF  FILE  ORGANIZATIONS 

1.1  Criteria  for  tterformance  Evaluation.  i 

There  are  three  important  criteria  commonly  used  to  evaluate  a file  i 

organization:  storage  cost,  processing  cost,  and  response  time.  The  i 

storage  cost  is  the  cost  of  storing  the  file  on  some  storage  medium;  the 
processing  cost  is  Incurred  in  doing  various  operations  on  the  file  and 
Includes  the  cost  of  utilizing  CPU,  core  memoiry,  I/O  channel,  other  I/O 
control  devices,  etc.;  the  response  time  is  the  time  elapsed  from  the 
receipt  of  a request  for  a certain  operation  to  the  end  of  that  operation. 

Basically,  the  storage  cost  is  fixed  and  the  processing  cost  is 
variable.  The  sum  of  these  two  should  be  as  small  as  possible.  On  the 
other  hand,  the  response  time  is  only  required  to  be  below  a certain  value. 
Generally,  more  storage  can  be  used  to  decrease  processing  cost  and  vice 
versa.  Also,  more  storage  and/or  processing  equipment  can  be  used  to 
reduce  response  time.  To  choose  a proper  file  organization  for  a logical 
file,  one  tries  to  minimize  the  total  cost,  while  keeping  the  response 
time  below  an  acceptable  value. 

Given  that  a file  is  stored  using  a certain  file  organization.  It  is 
relatively  easy  to  estimate  the  storage  cost.  The  portion  of  processing 
cost  that  is  fixed  is  also  relatively  easy  to  estimate.  The  variable  por- 
tion of  the  processing  cost  Is  proportional  to  the  equipment  cost  and  the 
elapsed  time  in  using  that  equipment.  Total  processing  cost  Is  the  sum 
of  processing  costs  on  all  the  equipments.  Usually,  the  toal  elapsed  time, 
or  response  time,  is  a good  Indicator  of  total  processing  time. 
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The  response  time  consists  of  actual  processing  tine  and  wait  time. 

If  any.  The  latter  Is  largely  dependent  on  the  environment  and  Is  very 
difficult  to  estimate  from  the  file  organization  alone.  Furthermore,  wait 
time  Is  relatively  unimportant  In  choosing  a file  organization.  Ignoring 
wait  time,  the  response  time  represents  the  total  processing  time. 

Due  to  Its  central  Importance  we  will  concentrate  on  the  determina- 
tion of  processing  time  In  evaluating  the  performance  of  a file  organiza- 
tion. 


1.2  Variability  In  Performance 

A file  organization  will  frequently  yield  different  processing 
times  In  different  situations.  For  example,  the  read  time  In  a sequential 
file  depends  on  the  relative  position  of  the  desired  record  In  the  file. 

It  takes  the  longest  time  If  the  desired  record  happens  to  be  the  last  one 
In  the  file. 

In  general,  the  variability  In  the  performance  of  a file  organization 
comes  from:  (1)  variability  In  the  file  properties  such  as  the  number  of 
records  in  a file,  the  specific  records  In  a file  (when  the  number  of 
records  Is  fixed),  the  key  value  distribution  of  potential  records,  the 
sizes  of  records  (if  not  fixed-length) , and  so  on;  (2)  variability  In  the 
environment,  for  example,  the  nature  of  other  activities  on  the  same 
device;  (3)  variability  In  the  history  of  operations  on  a file,  for  example. 
If  the  same  records  are  added  to  and  deleted  from  a hash  file  In  different 
sequences,  different  physical  files  will  result,  yielding  different  per- 
forman(  < s. 
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In  order  to  adequately  determine  the  performance  of  a file  organiza- 
tion, such  variability  must  be  explicitly  accounted  for  by  using  measures 


such  as  variance  and  higher  moments.  However,  In  most  applications  only 
averages  are  employed.  Sometimes,  values  for  best  and  worst  cases  are 
also  used.  Part  of  the  reason  for  such  practices  is  that  determination 
of  variance  and  higher  moments  Is  not  only  difficult  but  Is  almost 
impossible  In  many  situations. 

The  models  developed  In  this  report  permit  determination  of  expli- 
cit measures  of  variability  of  processing  time  for  Index  sequential  and 
some  simpler  files. 


I 
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2.  INDEX  SEQUENTIAL  FILES 


2.1  An  Index  sequential  file  consists  of  a data  file  and  a hierarchy 
of  Index  files.  We  will  restrict  ourselves  to  certain  Implementations 
of  the  Index  sequential  files  so  that  the  derived  models  will  cover  common 
Implementations  and  yet  will  be  specific  enough  to  be  useful. 

Each  level  of  Index  file  Is  divided  Into  a collection  of  Index 
subfiles  which  are  stored  as  ordered  sequential  files.  We  will  con- 
sider the  case  where  these  index  subfiles  are  ordered  physical  sequen- 
tial files. 

The  data  file  Is  divided  Into  a number  of  data  subfiles,  and  an 
Index  record  at  the  lowest  level  points  to  each  of  these  subfiles.  It 
is  not  necessary,  although  desirable,  in  many  cases  to  maintain  the  data 
file  such  that  the  data  subfiles  are  ordered  by  a certain  condition, 
which  may  or  may  not  coincide  with  the  ordering  condition  of  the  Index 
files.  A data  subfile  Is  stored,  then,  using  any  file  organization 
that  Is  appropriate. 

We  will  consider  the  following  implementation  of  the  data  file  that 
encompasses  all  the  common  Index  sequential  files.  Each  data  subfile  Is 
further  divided  Into  a "prime  data  subfile"  and  an  "overflow  data  subfile" 
which  are  both  sequential  files.  An  Index  record  at  the  lowest  level 
points  to  each  prime  or  overflow  data  subfile.  The  collection  of  all 
prime  data  subfiles  Is  called  the  "prime  data  file"  and  the  collection 
of  all  overflow  data  subfiles  is  called  the  "overflow  data  file".  Note 
that  the  Implementation  where  there  Is  no  overflow  data  file  is  a special 
case. 
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Referring  to  subfiles,  the  phrase  "of  the  same  kind"  will  be  used 
to  mean  "of  the  prime  data  file,  of  the  overflow  data  file,  or  of  an 
Index  file  at  a certain  level." 

2.2  Storage  Medium 

We  will  consider  files  stored  on  disks,  which  are  commonly  used  for 
on-line  systems.  Such  a storage  medium  Is  In  units  of  "packs",  each 
having  one  or  more  cylinders  and  a number  of  tracks  on  each  cylinder. 

On  each  pack  there  Is  one  read/write  assembly  that  Is,  at  any  moment  of 
time,  located  at  a particular  cylinder,  at  all  the  tracks  of  that  cylin- 
der, and  at  an  angular  position.  The  angular  position  changes  at  a 
constant  speed  which  Is  a characteristic  of  the  disk.  However,  the 
cylinder  position  does  not  change  except  possibly  to  read  or  write  a 
record.  The  assembly  must  move  to  the  correct  cylinder  and  the  correct 
angular  position  to  read  or  write  a record.  The  packs  operate  Indepen- 
dently of  each  other. 

Note  that  this  description  Includes  the  head-per-track  disk  devices 
as  a special  case  where  the  time  to  mover  from  one  cylinder  to  another 
may  be  considered  to  be  always  zero. 

Thus,  the  time  to  read  or  write  a record  at  an  address  Is  the 
sum  of  a seek  time  (to  position  to  the  cylinder) , a latency  time  (to 
position  to  the  angular  position) , and  a transmission  time  Old  actually 
read  or  write  the  record).  The  seek  time  Is  a function  of  the  cylinder 
addresses  of  the  last  record  accessed  on  the  same  pack  and  the  desired 
record.  The  latency  time  Is  a function  of  the  angular  position  of  read/ 
write  assembly  at  the  time  when  It  arrives  at  the  correct  cylinder  and 
the  desired  angular  position.  The  track  address  of  the  desired  record  is 


Irrelevant  for  timing  purposes. 


3.  CONVENTIONS  AND  NOTATIONS 


3. 1 Subscripted  Variables 

The  following  variables  may  have  subscripts  listed  later. 

A add  time 

0 delete  time 

K seek  time;  time  to  position  the  read/write  assembly  of 

a disk  pack  to  a specific  cylinder 

L latency;  time  to  position  the  read/write  assembly  to 

the  start  of  a record 

M relative  position  (a  number)  of  a record  In  a file 

after  operation 

the  number  of  records  shifted  In  a physical  sequential 
file  In  an  operation 

N the  number  of  records  In  a file.  When  referring  to  a 

physical  file.  It  may  or  may  not  Include  deleted  records 
In  It. 

R read  time;  time  to  read  a record 

T transmission  time;  time  to  transmit  a record.  Including 

the  time  to  pass  a gap  between  records, 

U update  time;  time  to  update  a record 

3.2  Non-subscrlpted  Variables 

H search  time;  time  to  read  and  examine  a number  of  records 

In  order  to  find  a record  meeting  a given  condition. 
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p a probability 

Q number  of  levels  In  an  Index  sequential  file 

r a probability 

3.3  Subscripts 

The  following  symbols  are  used  as  subscripts  to  denote  that  the  sub- 
scripted variables  pertain  to  a file  or  a certain  file  organization  or  a 
file  of  a certain  kind  in  an  Index  sequential  file. 

D direct  file 

F an  Index  file  at  any  level,  a prime  data  file  or  an 

overflow  data  file 
1 an  Index  file  at  level  1 

Numerals  1,^ etc  ' fndex  file  at  the  Indicated  level 
P prime  data  file 

Q the  top  level  Index  file 

S a sequential  file 

V an  overflow  data  file 

X an  Index  sequential  file 


3.4  A Special  Notation 


The  notation 


I 


My 

Y or  ^ Y Is  used  to  denote  the  sum  of 
1 


random 
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variables  which  are  Identically,  Independently  distributed  as  Y,  where 
may.  In  general,  be  any  Integer  and  Y Is  a given  random  variable. 

That  Is , y Y = ^ Z , where  each  Z . Is  distributed  as  Y and  all  Z . ' s 

are  Identically,  Independently  distributed  and  Z^'s  are  not  explicitly 


referred  to. 


4.  DEVELOPMENT  OF  BASIC  EXPRESSIONS 


In  this  section  we  develop  basic  expressions  for  read,  add,  delete 
and  update  operations  on  an  Index  sequential  file. 


4.1  Read  Time 


The  read  operation  consists  of  two  steps: 

Cl)  Search  In  each  level  of  the  Index  files  for  the  address 
of  the  desired  data  subfile;  and 
Cll)  Search  In  the  prime  data  subfile  or  the  overflow  data 
subfile  as  Indicated  by  step  (1) . 

The  expressions  for  the  times  spent  In  these  steps  are  derived 


below. 

The  search  for  the  address  starts  In  the  top-level  (level  Q) 

Index  file.  The  time  spent  here  consists  of  seek  time  K^,  latency 
time  Lq  and  transmission  time  for  records,  ^q^q*  The  quantities 
Kp,  Lp  and  Mp  are  random  variables  and  hence  the  time  spent  at  level 
Q Is  a random  variable  given  by  (Kp  + Lp  + Mp*Tp).  Note  that  the 
Individual  terms  In  this  sum  may  or  may  not  be  statistically  Independent. 

Next,  we  search  at  levels  (Q-1) , (Q-2),  ...,  1.  The  time  spent 
at  level  1 Is  given  by  (K^  + Henc^  the  total  time  spent 

In  the  Index  files  Is 


TTSIF 


■ j.  »i  ^ '•i  Vi) 


With  probability  p,  we  search  for  the  record  in  a prime  data 
subfile,  a physical  sequential  file.  The  time  spent  here  consists 
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of  seek  time  K^,  latency  time  Lp  and  read  time  for  Mp  consecutive 
records,  Thus,  the  total  time  spent  In  the  prime  data  subfile 

Is  given  by  the  random  variable 

TTSIPDS  = (Kp  + Lp  + MpTp)  (2) 

I 

With  probability  q,  we  search  for  the  desired  record  in  an 
overflow  data  subfile, which  is  a pointer  sequential  file.  The  time 
spent  here  is  the  sum  of  seek  time  and  the  time  to  read  records 
in  the  file,  ^ (L^  + T^).  The  latter  item  is  the  sum  of  identically 
independently  distributed  random  variables.  The  total  time  is  thusT 

+ (3) 

When  we  combine  the  above  expressions,  the  read  time  for  a 

single  record  is  given  by  the  random  variable 

Q 

R = I + Li  + M^T^)  (4) 


p:  Kp+Lp+MpTp 

q:  Ky  + pLy  + T^) 


where  the  isecond  term  is  a discrete  mixture  of  two  random  variables. 
A definition  of  discrete  mixtures  is  given  in  Appendix  I . 


4. 2 Add  Time 

The  add  operation  consists  of  two  steps: 

(i)  Search  the  index  files  to  determine  the  address  of  the  data  sub- 
file where  the  given  record  should  be  added,  and 
(ii)  Add  the  given  record  to  the  data  subfile  as  indicated  by  step  (i). 
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The  expressions  for  the  times  spent  in  these  steps  are  derived  below. 

(a)  The  time  spent  in  step  (1)  is  the  same  as  for  step  (1)  in 
the  read  operation  and  Is  given  by  Equation  (1). 

(b)  With  probability  p,  the  given  record  Is  added  to  a prime 
data  subfile.  Let  the  added  record,  after  the  add  operation,  be  the 
Hpth  record  of  the  subfile. 

In  order  to  maintain  ordering,  a number,  M^,  of  records  must 
be  moved  to  make  space  for  the  given  record.  Note  that  Is  a 
random  variable  taking  values,  0,  1,  2,  ...,  Np,  where  Np  Is  the 
number  of  records  In  a prime  data  subfile  and  may  also  be  a random 
variable.  Note  that  the  time  needed  to  replace  one  record  In  a 
sequence  of  consecutive  records  Is  1 + Tp.  If  there  Is  space  In  the 
prime  data  subfile  to  accommodate  the  given  record,  and  only  one 
record  In  addition  to  the  given  record  can  be  held  In  core,  the  time 
taken  to  move  records  Is  (1  + Tp)M^.  However,  If  the  size  of  the 
prime  data  subfile  Is  limited  and  the  added  record  will  cause  the 
limit  to  be  exceeded,  one  or  more  records  may  have  to  be  moved  to 
other,  perhaps  newly  created,  prime  or  overflow  data  subfiles.  The 
time  required  for  this  case  will  be  governed  by  the  specific  tech- 
nique used  to  handle  such  situations.  Here,  we  consider  only  the 
case  where  there  Is  enough  room  In  the  prime  data  subfile  to  accommo- 
date the  new  record. 

The  total  time  spent  In  the  prime  data  subfile  Is,  thus,  the 
sum  of  seek  time  Kp,  latency  time  to  reach  the  first  record  Lp, 
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search  time  to  read  Mp  records  Mp*Tp,  and  time  to  move  records 

(1  + Tp) , and  Is  given  by 

TTSPDS  - (Kp  + Lp  + Mp-Tp  + M^(l  + Tp).  (5) 

Cc)  With  probability  q,  the  given  record  is  added  to  an 

overflow  data  subfile,  which  is  a pointer  sequential  file,  where 

adding  a given  record  between  two  consecutive  records  A and  B 

CB  follows  A)  requires  A to  be  updated  and  the  given  record  written. 

The  time  needed  to  add  a record  to  an  overflow  data  subfile 

M^-1 

consists  of  seek  tlme^K^,  search  time  for  K^~l  records,  1(1*^  "*■  » 

time  to  read  record  and  update  record  My-1»  1.  time  to  write  the 
added  record, + T^  . 

The  total  time  spent  here  Is  thus  given  by 

TTAODS  = Ky  + (Ly  + Ty)  + 1 + (Ly  + Ty)  (6) 

Note  that  if  there  is  a backward  pointer,  extra  time  will  be 
needed  for  updating  the  next  record  and  should  be  Included  In  the 
above  time. 

Combining  the  expressions  from  (a),  (b)  and  (c),  the  time 
for  adding  a single  record  under  the  conditions  described  above  Is 
given  by  the  random  variable 

p:  Kp  + Lp  + MpTp  + M^(l  + Tp) 

q:  Ky  + r (Ly  + Ty)  + 1. 

where  the  second  expression  is  a discrete  mixture  of  two  random 
variables  (see  Appendix  I). 


1-1 


(K^  + 


Vi> 


C7) 
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4.3  Update  Time 

The  update  operation  consists  of  reading  the  record  to  be  updated 
followed  by  writing  over  It  the  updated  record.  The  total  time  Is  given 
by  the  random  variable  U, 

U - R + 1 (8) 

where  R Is  the  read  time  given  In  Section  4.1. 

4.4  Delete  Time 

The  delete  operation  consists  of  reading  the  record  to  be  deleted 
and  writing  over  It  a deleted  record.  This  requires  one  read  operation 
and  one  disk  revolution.  Hence,  the  delete  time  Is  given  by  a random 
variable 

D ■=  R + 1 (9) 

where  R Is  the  read  time  given  In  Section  4.1. 

An  alternative  to  this  delete  operation  Is  to  remove  the  record  from 
the  subfile,  thus  decreasing  the  number  of  records  In  the  subfile  by  1. 

The  extra  time  needed  to  remove  the  record  can  be  obtained  by  using  an 
approach  similar  to  the  one  used  for  obtaining  read  and  add  times  In 
Section  4.1  and  4.2,  respectively.  This  time  Is  not  reflected  In  the 
above  expression. 
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5.  NOTES  ON  THE  APPLICATION  OF  THE  BASIC 
MODELS  - SPECIAL  CASES 


The  purpose  of  this  section  is  two“fold! (1)  to  derive  timing  expres*" 
slons  for  some  useful  special  cases,  and  (2)  to  point  out  how  the  basic  ex- 
pressions for  the  index  sequential  file  are  modified  for  specific  purposes. 

Each  of  these  cases  specifies  one  aspect  of  the  general  index  sequen- 
tial file.  If  two  or  more  aspects  are  specified  at  a time,  or  some  other 
aspect  is  specified,  this  section  should  be  helpful  in  showing  the  general 
approach  to  be  used  for  the  given  situation. 


5.1  Index  Sequential  File  on  a Single  Pack 

If  an  index  sequential  file  is  stored  on  a single  pack,  then  in  order 
to  carry  out  a series  of  operations  on  random,  individual  records,  the 
read/write  assembly  moves  only  from  the  top  level  index  file  to  index  files 
at  levels  (Q-1) , (Q-2) , , , . , 1 and  then  to  either  an  overflow  or  a prime 
data  subfile  and  back  to  the  top-level  index  file.  In  this  case  the  seek 
time  to  each 'kind "Cas  defined  in  Section  3,3)  of  file,  say  to  an  index  file 
at  level  1,  is  more  conveniently  denoted  by  K2  indicating  the  seek 

time  from  a record  in  an  index  subfile  at  level  2 to  the  beginning  of  an 
index  subfile  at  level  1.  The  quantities  p,  y,  q,  Kp  q, 

etc,  are  similarly  defined  as  the  seek  times  for  a record  in  a subfile 
of  a certain  kind  to  the  beginning  of  a subfile  of  another  kind. 

The  seek  time  to  the  top-level  index  file  needs  some  clarification. 
Since  the  random  variable  is  dependent  on  whether  the  last  record  is  in  the 
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prime  data  file  or  the  overflow  data  file.  It  le  best  separated  Into  two 
terms  Kp  ^ and  q and  combined,  respectively,  with  the  times  spent  In 
the  prime  and  the  overflow  data  files. 

Using  arguments  similar  to  those  for  expression  (4)  In  the  basic 
model,  the  read  time  for  an  Index  sequential  file  residing  on  a single 
pack  Is  given  by  the  random  variable  ' 


L„  + M„-T^  + I (K 
1=1 


q Q 


+ L + M •!  ) 
1+1,1  1 1 1^ 


“v 


(10) 


Similarly,  the  add  time  for  an  Index  sequential  file  residing  on  a 
single  pack  Is  given  by  the  random  variable 


fp:  K^^p  + Lp  + Mp.Tp+M;.(l  + Tp)  +Rp^Q 

I My+1  (11) 

l<i-  s+  r ts,  + V ‘ 'S/.Q 


The  relationships  between  update  and  delete  times  and  read  time  are 


the  same  as  In  the  basic  models. 


5.2  Number  of  Records  Searched  In  a Subfile 


The  number  of  records  ttiat  must  be  read  before  the  desired  record 
Is  found  In  a subfile  Is,  In  general,  a random  number. 

Consider  a subfile  of  kind  X,  where  X > (an  Integer  between  1 and  Q), 
P,  or  V,  Indicating  an  Index  subfile  at  a certain  level,  a prime  data 
subfile,  or  an  overflow  data  subfile,  respectively.  If  the  desired  record 
exists  In  the  subfile,  and  If  all  records  In  a subfile  of  kind  X have 
equal  probability  of  being  used  for  read,  add,  delete  or  update  operations, 
then  the  number  of  records  searched,  M , has  a discrete  uniform  distribution 
over  the  range  C1,N  ),  where  N Is  the  number  of  records  in  this  subfile. 

On  the  other  hand.  If  the  desired  record  does  not  exist  In  the  subfile, 
all  the  records  will  have  to  be  searched  before  the  operation  is  over,  and 
hence 

Let  r^  be  the  probability  that  the  desired  record  exists  In  subfile  X. 
Then  Is  given  by  the  following  discrete  mixture: 

frjj,  : 1,  1 = 1,2,  ..., 

M,=  (12) 

[_1  - r^:  Njj 


5.3  Interference  from  Other  Files 

If  there  are  other  files  that  share  the  same  pack.  It  Is  possible 
that  the  pack  has  to  leave  a cylinder  of  the  Index  sequential  file  under 
consideration  and  return  to  It  later.  Effectively,  the  seek  time  is 
Increased  as  described  below. 
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Let  F denote  all  other  fllea  that  share  the  same  device  with  a file 
of  kind  X,  where  X - (an  Integer  between  1 and  Q) , P or  V.  The  random 
variable  Is  the  sum  of  (1)  seek  time  from  a file  of  kind  X to  a file 
of  F,  (11)  the  time  spent  In  a file  of  F,  and  (ill)  the  seek  time  from 
F to  X.  The  distributions  of  the  above  three  variables  may  be  estimated 
by  noting  the  distributions  of  the  appropriate  cylinder  addresses  and  the 
overall  processing  time  In  a file  of  F. 

5.4  Mixed  Operations 

If  a series  of  different  operations  are  carried  out  on  the  Index 
sequential  files,  the  time  to  process  a single,  random  record  Is  given  by 
the  discrete  mixture  (pj^:R.  P^'A.  Py^U,  Pjj:D),  where  Pj^,  p^,  Py,  and  Py 
are  the  probabilities  of  read,  add,  update  and  delete  operations,  respec- 
tively. Each  operation  may  have  Its  own  distributions  of  the  numbers  of 
records  searched,  cylinder  addresses  (for  seek  times),  etc.  This  Is 
easily  accounted  for  by  considering  the  discrete  mixtures  of  the  appro- 
priate seek  times  in  different  situations. 


6.  TIMING  EXPRESSIONS  FOR  SEQUENTIAL  FILES 


Since  an  Index  sequential  file  Is  composed  of  a number  of  sequen- 
tial files,  the  same  approach  used  before  In  deriving  the  timing  expres- 
sions for  Index  sequential  files  may  be  applied  to  sequential  files  to 
obtain  timing  expressions. 

The  timing  expressions  for  read,  add,  delete,  and  update  opera- 
tions on  sequential  files  are  given  In  this  section,  along  with  some 
notes  on  applying  these  expressions  to  actual  files. 

6.1  Read  Time 

The  time  to  read  one  record  from  a sequential  file  Is  equal  to 
the  sum  of  (1)  seek  time  K and  (11)  search  time  H to  read  and  examine 
M records  until  the  desired  record  is  read  or  until  every  record  Is  read. 
Thus,  the  read  time  R = K + H. 

The  seek  time  K Is  determined  by  the  cylinder  address  (in  general, 

a random  variable)  of  the  read/write  assembly  before  the  read  operation 

and  the  cylinder  address  of  the  first  record  In  the  file. 

The  search  time,  H,  Is  different  for  physical  sequential  and  pointer 

sequential  files.  For  the  former,  search  time  is  given  by  H «=  L + MT, 

where  L Is  latency  to  reach  the  first  record  and  T Is  the  transmission 

time  of  a record  and  may  also  be  a random  variable  If  the  record  length 

Is  not  fixed.  For  a pointer  sequential  file,  search  time  is  given  by 

H = ^(L  + T),  where  L Is  the  latency  to  reach  the  first  read  or  the 

M 

next  record  Indicated  by  the  pointer  field  and  T Is  the  transmission 
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time  as  before 


The  latency,  L,  Is  a function  of  the  position  of  the  read/wrlte 
assembly  relative  to  a desired  record  and  may  usually  be  approximated 
by  a uniform  random  variable  In  the  Interval  (0,1),  provided  the  timing 
unit  Is  the  revolution  time  of  the  disk. 

Note  that  M Is  determined  by  (1)  the  probabilities  of  reading  the 
records  In  the  sequential  file,  (11)  the  probability  of  the  desired  record 
being  In  the  file,  and  (111)  whether  the  records  are  ordered  by  a certain 
condition. 

1 

6.2  Add  Time 

The  add  time  Is  the  sum  of  Cl)  time  to  locate  the  address  where  the 
new  record  is  stored,  (11)  the  time  to  store  the  record  at  the  found 
address,  and  (ill)  the  time  to  adjust  the  file  to  maintain  the  Integrity 
of  the  file. 

There  are  many  variations  of  the  add  operation,  A common  case  Is 
considered  here  as  an  example.  It  Is  assumed  that  (1)  the  records  In 
the  file  are  ordered  by  a key,  (11)  the  records  are  deleted  from  the  file 
by  modifying  them  to  a special  record,  (111)  when  the  limit  to  the  number 
of  records  that  can  be  stored  In  the  file  Is  exceeded,  a new  sequential 
file  Is  created  elsewhere  and  the  new  record  is  stored  there,  and  (iv) 
only  one  record  in  addition  to  the  new  record  may  be  held  In  core. 

Let  K and  L be  the  seek  and  latency  times,  respectively,  to  prepare 
for  reading  the  first  record,  N be  the  number  of  records  (excluding  deleted 
records)  In  the  file  before  addition,  M be  the  record  number  of  the  new 
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record  (including  deleted  records)  and  M'  be  one  less  the  number  of  records 

between  the  Mth  record  before  addition  and  the  first  deleted  record  after 

it  Conclusive).  Thus,  if  the  new  record  must  be  added  between  records  2 

and  3 ClncluJing  deleted  records)  to  satisfy  the  ordering  condition,  and 

record  3 is  not  a deleted  record  but  record  A is,  then  M = 3 and  M'  = 1. 

The  total  time,  shown  in  the  breakdown  given  above,  is  (0)  K + L + ^T 

M 

Cll)  1,  and  cm)  I T + P*  C ^ where  p is  the  probability  that  the 
M'  1-p:  0 

capacity  is  exceeded  and  C is  the  time  to  create  a new  file  containing  the 
last  record  from  the  original  file.  Thus 

f PS  C 

A=It+1+^T+  (13) 

M M'  i^l-p:  0 


6. 3 Update  Time 

The  update  time  is  given  by 

U = R + 1.  (lA) 

6. A Delete  Time 

The  delete  time  varies  with  the  delete  operation.  For  the  case  where 
a record  to  be  deleted  is  simply  marked  so,  the  delete  time  is  equal  to 
update  time  and,  hence, 

D = R + 1.  C15) 
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7.  TIMING  EXPRESSIONS  FOR  DIRECT  PILES 


The  read  time,  R,  Is  the  sum  of  (1)  seek  time  K and  latency  time 
L before  reaching  the  desired  record  and  (11)  transmission  time  T. 

The  seek  time  K Is  a function  of  the  number  of  cylinders  crossed, 
which  Is  the  absolute  difference  between  the  cylinder  Address  C (a  random 
variable)  of  the  read/write  head  at  the  beginning  of  the  add  operation 
and  the  cylinder  address  C'  (another  random  variable)  of  the  desired 
record.  Let  s(x),  a characteristic  of  the  disk  device,  be  the  seek 
time  to  cross  x cylinders.  Then  the  read  time  Is 

R - s(|c  - C I)  + L + T.  (16) 

The  add  time  Is  the  Sum  of  (1)  the  read  time  for  the  existing  record 
at  the  given  address,  and  (11)  the  time  to  write  the  given  record  at  the 
given  address  If  allowed.  Thus, 

A - R + 1 (17) 

The  update  time  Is  U ■■  R + 1,  and  the  delete  time  Is  D - R 1, 
since  a record  to  be  deleted  Is  simply  updated. 
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8.  METHODS  FOR  NUMERICAL  TRANSFORMATIONS 
OF  RANDOM  VARIABLES 

In  the  models  derived  In  the  previous  sections  various  transfor- 
mations of  random  variables  are  Involved,  such  as  the  summation  of  a 
number  of  random  variables  (some  discrete,  some  continuous),  the  dis- 
crete mixture,  and  the  summation  of  N random  variables  where  N Is  also 
a random  variable.  There  are  no  existing  methods  which  are  simple  and 
flexible  enough  to  make  the  application  of  these  models  practical. 

The  purpose  of  this  section  Is  to  discuss  various  computational 
methods  possible  for  this  purpose  and  to  find  a computational  strategy 
that  (1)  minimizes  the  computational  efforts,  (11)  can  be  used  for 
cases  where  complete  or  partial  data  about  the  component  random  variables 
Is  available,  and  (111)  can  work  with  totally  arbitrary  distributions 
that  may  be  discrete,  continuous,  or  mixed. 


8.1  Approximating  Density  Functions 

In  many  cases  the  values  of  the  density  function  of  each  random 
variable  may  be  approximated  at  a certain  number  of  abscissa  points. 

The  disadvantages  of  this  approach  are  (1)  the  density  function  of  each 
random  variable  must  be  estimated  In  great  detail,  and  (11)  a large  amount 
of  computation  and  storage  Is  required  In  many  situations  (e.g.  a very 
Irregular  discrete  density  function.) 

Sometimes  flexible  distributions,  such  as  beta  distribution,  may 
be  used  to  fit  a density  function.  This  method  Is  limited  to  rather  well 
behaved  distributions  and  cannot  be  applied  to  totally  arbitrary  density 
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functions.  Furthermore,  transformations  other  than  summation  are  dlffl- 
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cult  and  may  Introduce  different  (e.g.,  non-beta)  distributions.  There- 
fore, they  are  not  suitable  for  the  type  of  situation  encountered  in  the 
present  context. 

8. 2 Histogram  Method 

Another  possible  method  Is  to  use  a step  function,  called  "histogram 
function",  to  approximate  a density  function.  Given  a set  of  points 

a(l),  a(2) a(n),  the  histogram  function  Is  a step  function,  which  Is 

chosen  such  that  the  Integration  between  a(l)  and  a(l+l)  of  the  histogram 
function  Is  equal  to  that  of  the  approximated  density  function. 

A procedure  to  obtain  an  approximate  histogram  function  for  the  sum 
of  two  random  variables,  whose  Individual  histogram  functions  are  given. 

Is  described  In  Appendix  II. 

Various  difficulties  and  Inaccuracies  are  Introduced  by  the  use  of 
this  method.  First,  not  every  density  function  can  be  represented  exactly 
by  a histogram  function.  Secondly,  the  convolution  of  two  histogram  func- 
tions cannot  be  represented  exactly  by  a histogram  function.  Cit  is  a 
piecewise  linear  function.) 

In  order  to  test  the  feasibility  of  this  method,  a number  of  standard 
normal  random  variables  were  added  (see  Appendix  II) . The  errors  intro- 
duced by  the  method  seem  to  Increase  linearly  with  the  accumulative  number 
of  additions  performed  In  a histogram.  This  Is  generally  satisfactory. 
However,  If  the  interval  size  Is  kept  constant,  the  number  of  Intervals 
Increases  multlplicatlvely  with  each  addition  performed,  even  if  Inter- 
vals with  small  weights  are  discarded.  If  the  Interval  size  Is  Increased 
after  each  addition,  more  complicated  procedures  must  be  used  and  greater 
error  will  be  Introduced,  A detailed  discussion  of  this  aspect  Is  given 
In  Appendix  II. 
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The  histogram  method  Is  applicable  for  discrete  distributions 
which  are  Irregular,  but  requires  a large  amount  of  storage  in  many 


cases.  Although  the  histogram  method  has  the  advantage  of  being 
applicable  for  cases  where  only  limited  information  is  available,  it 
was  abahdoned  in  this  study  because  the  method  becomes  much  more 
complicated  for  transformations  other  than  addition  and  the  computation 
efforts  can  become  too  costly. 

Various  other  methods  of  approximating  a function  exist  but  are 
not  useful  because  we  must  be  able  to  handle  totally  arbitrary  func- 
tions that  may  be  continuous,  discrete,  or  mixed. 

8,3  Moments  Method 

It  is  well  known  that  a random  variable  is  completely  defined  if 
all  of  its  moments, if  existing,  are  known.  Also,  if  only  the  first 
few  moments  are  known,  partial  information  about  the  shape  of  the  dis- 
tribution may  be  deduced.  If  the  transformations  may  be  conducted  solely 
on  the  first  few  moments,  the  method  may  be  used  to  gain  information 
about  a random  variable  depending  on  the  Information  available.  If 
more  moments  are  needed,  one  simply  tries  to  obtain  more  information 
(in  terms  of  more  moments)  about  the  component  random  variables  and 
applies  the  method  again. 

Formulas  for  conducting  various  random  variable  transformations  j 

on  the  moments  are  presented  in  Appendix  III.  The  transformations 
Include  summation,  multiplication,  and  discrete  mixture,  which  are  ' | 

needed  for  the  index  sequential  file  model  derived  previously.  . ■' 

i 


Once  the  moments  of  a random  variable  are  obtained,  various 
statistics  such  as  the  mean,  variance,  skewness,  kurtosls,  etc.  may 
be  estimated  readily.  In  addition,  the  approximate  density  function 
may  be  estimated  from  the  first  four  moments  and  estimations  of  the 
density  function  can  be  obtained  (see,  for  example,  [Royden  1953] 
and  [Elderton  1969]). 

The  moments  method  has  the  advantages  that  the  computation 
procedure  Is  simple  and  the  method  Is  flexible  In  that  the  number  of 
moments  used  can  be  varied  according  to  available  data  and  desired 
accuracy. 

The  moments  method  of  carrying  out  random  variable  transfor- 
mations Is  most  appropriate  for  the  Index  sequential  file  models 
derived  In  the  previous  sections.  This  method  will  be  used  In 
several  numerical  examples  to  Illustrate  the  use  of  the  models. 
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Appendix  I 
DISCRETE  MIXTURES 


1.  Definition 

The  "discrete  mixture"  Y of  n statistically  Independent  random 

n 

variables  . . .X^,  with  weights  of  Pj»Po»*«*tP  t I P^"l  a random 

1-1  ^ 

variable  whose  density  function  f^  Is  given  by  the  weighted  sum  of  the 

density  functions  f of  the  "components"  X,,  X,,  ....  X . In  other 

1 i / n 

n 

words,  fyCX)  - J]  p f (X).  The  following  are  notations  for  a dls- 
1=1  ^ *1 

Crete  mixture: 

Y = (Pj^:X^;  1-1 n) 

Y - (Pj^tXj^,  P2'^2*  **•’  Pn*\^ 

This  definition  also  can  be  easily  modified  for  the  discrete 

random  variables  X,,  X„,  X . 

1 z n 

2.  Illustration 

Informally,  Y is  a random  variable  which  assumes  the  behavior  of 

Xj^  with  probability  p^,  for  i = 1,2,  ...,n.  Note  that  it  is  different 

n 

from  the  weighted  sum  of  X. *s,  namely,  Z = I p X whose  density  function 

1=1  1 

is  the  convolution  of  density  functions  of  ^ l,2,...,n. 

Let  Xj^  = 1 with  probability  0,5  and  X^  = 2 with  probability  0,5 
and  - 10  with  probability  0,5  and  X2  = 20  with  probability  0.5.  Note 
that  Xj^  and  X2  are  discrete  mixtures  of  degenerate  random  variables, 

Let  Y be  a discrete  mixture  of  and  X2  with  weights  0,2  and  0,8, 
i,e,,  Y - G).2;Xj^,  0,8:X2)  and  let  Z = 0.2(Xj^)  + 0,80X2);  then  Y = 

CO.l:  1,  0.1:  2,  0.4:10,  0.4;  20)  and  Z = 00,25:  8.2,  0.25:8.4,  0.25:  16,2, 
0.25:16.4).  Clearly,  they  are  different  random  raviables.  The  means  of 
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Y and  Z are  equal,  but  hlghre  moments  are  not. 

The  following  theorems  can  be  easily  proven  by  directly  using  the 
definitions. 


3.  Theorem  A-1 

The  moments  of  a discrete  mixture  are  the  weighted  sums  of  the 
corresponding  moments  of  Its  components,  l.e. 
n 


= 


I 

3=^1 


Pi  Vx^Ci) 


where 

Y *=  (Pji  X^;  j = 1,2,  ...,n)  and  UyCD  and 
moments  about  the  origin  of  Y and  X.  respectively. 


4 .  Theorem  A-2 

Z + (p^:X^;  1 = 1,2,  . . . ,n)  Is  Identical  to  (p^;  Z + X^;  1 = 1,2,.. 
If  Z and  X^,  1 = l,2,...,n  are  all  statistically  Independent. 


5.  Theorem  A-3 

( Pj^:X,  ('ll*  ‘I2’  ^ ^1"  ^2**^1*'  **2*^2'  ^ ^ where 

X,  Y and  Z are  statistically  Independent. 
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Appendix  II 

USING  STEP  FUNCTIONS  FOR  CONVOLUTION 


1.  Introduction 

In  order  to  carry  out  the  transformations  Indicated  In  the  evalu- 
ation model  of  Index  sequential  files,  simple  and  convenient  ways  have 
to  be  chosen.  In  this  appendix,  we  will  Investigate  the  method  of 
using  step  functions,  called  "histogram  functions",  to  approximate 
density  functions  by  studying  the  sum  of  Independent  random  variables. 

A 'histogram  function"  hA(l}  of  a random  variable  A gives  the 
probability  that  A lies  In  Interval  (a[l],  atl+1])  for  1 ■ l,2,...,m, 
where  a[l],  a[2],...,  a[iiri-l]  are  parameters  of  the  histogram  function 
an^  constitute  the  'hode  set".  A specific  value  hA(l)  for  a given  1 
Is  called  the  "weight"  of  the  histogram  function  In  the  Interval 
(a[l],  a[l+l]),  or  at  node  a[l].  A random  variable  whose  density 
function  Is  a step  function  Is  called  a "histogram  variable". 

We  assume  the  nodes  In  the  node  set  are  all  unequal  and  are  num- 
bered In  Increasing  order,  and  hA[l]hA[iiri-l]  ^ 0,  without  loss  of 
generality. 

We  further  assume  that  the  nodes  are  all  finite.  If  either  or 
both  of  the  end  nodes,  a[l]  and  atnri-l],  are  Infinite,  a histogram  function 
can  still  be  considered  an  approximation  to  the  true  density  function. 


9-29 

J 


but  It  causes  problems  In  transformations. 

Throughout  this  appendix,  the  term  "histogram"  will  be  used  as 
an  abbreviation  for  "histogram  function".  Being  an  approximation  to 
a density  function,  a histogram  contains  only  partial  Information  about 
a random  variable.  Therefore,  It  is  Impossible  to  obtain  the  convolu- 
tion of  two  random  variables  given  only  their  histograms. 

The  convolution  of  two  histograms  Is  not  a histogram.  However, 
after  choosing  a node  set,  one  can  always  obtain  a histogram  as  an 
approximation  to  the  convolution  of  the  two  density  functions. 

Such  an  approximate  convolution  contains  two  sources  of  errors, 
the  first  being  approximating  the  original  random  variables  with  histo- 
grams and  the  second  being  approximating  the  convolution  of  histograms 

with  a histogram. 

f 

[»  The  following  example  shows  some  of  the  problems  involved  in 

trying  to  find  out  the  distribution  of  the  sum  of  two  random  variables 
given  only  their  histograms.  Later,  we  will  study  the  errors  on  a 
test  case  using  normal  variables. 

2.  An  Example 

Suppose  the  following  histograms  are  given  for  two  statistically 
independent  random  variables  A and  B.  The  true  density  functions  are 
^ irrelevant  for  this  example. 
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The  following  table  gives  Information  about  the  joint  distribu- 
tion of  A and  B and  hence  about  the  distribution  of  C *■  A -f-  B.  Each 
entry  shows  three  Items,  p(l,j),  8(1, j)  and  u(l,j),  where  p(l,j)  Is 
the  probability  that  A Is  In  (a[l],  a[l+l])  and  B Is  In  (b[j],  b[j+l]) 
and  8(l,j)  and  u(l,j)  are  the  numbers  enclosed  In  parentheses  repre- 
senting the  lower  and  upper  bounds  of  corresponding  values  of  C.  Each 
p(l,j)  Is  also  the  conditional  probability  that  C Is  In  (all]+b[j], 
a[l+l]+b[j+l]) , given  that  A and  B are  In  the  Indicated  Intervals, 
tespectlvely.  The  headings  Indicate  a[l]  and  b[j]. 

Note  that  the  Invervals  of  C,  (s(l+j),  u(l+j))  overlap. 
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TABLE  C“1  p(i»i)  and  (s(i»j)/  u(i»j)) 
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Since  only  partial  information  la  given  for  the  distributions  of 
A and  B,  it  is  possible  only  to  obtain  an  approximate  distribution  for 
C.  However,  if  A*  and  B*  represent  the  random  variables  defined  by 
the  histograms  given  above,  then  the  sum  C'  “ A*  + B'  can  be  obtained 
exactly.  We  will  try  to  obtain  a histogram  from  C'  as  an  approxima- 
tion to  the  density  function  of  C. 

Let  AflJ  be  a uniformly  distributed  random  variable  over  the 
interval  [a(i],  a[i+l])  and  B(l)  be  a uniformly  distributed  random 
variable  over  the  interval  b[l+l]).  Also,  let  C[l,j]  = A[l]  + BlJ], 

Consider  the  sum  of  A[l]  and  B[l],  which  are  over  intervals 
(-2,-1)  and  (2,4),  respectively.  Let  C[l,l]  = A[l]  + B[l].  Graph- 
ically, the  density  functions  A[l),  B[l],  and  C[l,l]  are  shown  as 
follows : 
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In  Table  C-1,  C[l,l]  contributes  (0,04) (0.25)  - 0.01  to  interval 
(0,1)  for  C,  (0.04)0.5  = 0.02  to  (l,2)and  (0.04)  (0.25)  = 0.01  to  (2,3). 

The  weights  of  each  C[i,j]  can  be  similarly  broken  down.  If 
we  sum  up  all  the  weights  for  each  interval,  the  following  histogram 
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3.  A General  Procedure  for  a Simple  Case 

Let  hA(i)  on  a[i],  1 = 0,  1 n,  and  hB(j)  on  b[J ] , j = 0,  1, . . . ,m 

be  the  two  histograms  for  random  variables  A and  B,  respectively.  Then, 

using  the  notations  for  discrete  mixture  in  Appendix  A,  we  define 

A'  = (hA(l):  Ati],  1 = 0,1 n) , and  B’  - (hB(j):  B[jl,j=0,l,2 m) 
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where  All]  and  B[J]  are  uniformly  distributed  oyer  (a[l],  a[l+l]) 
and  (b[J],  bIj+1]},  respectively.  A*  and  B*  are  approximations  to 
A and  B,  respectively. 

Using  theorems  derived  in  Appendix  A,  we  have  C’  = A*  + B’  *» 
(hA(l)hB(j)  : A[i]+B[Jl). 

To  obtain  an  approximation  C*  to  C •=  A + B,  (1)  choose  a node 
set  for  C' , and  then  (2)  compute  hC*(i)  from  C’. 

If  a[l]  = 1,  and  b(j]  = j,  and  we  choose  c[k]  “ k,  k = o,  1,  ...» 

(mtn),  then  C - (hA(i)hB(j)  : C[i,J]),  where  C[i,j]  - A[l]  + B[j] 
and  Pr(C[l,J]  Is  In  (1+j,  1+j+l)  ! A[i]  Is  In  (1,  1+1)  and  B[j]  Is 
in  (J,  j+1)  ) = 0.5hA(l)hB(j),  and  Pr(  Cti.j]  Is  In  (i+j+1,  i+j+2)  ! 

A[l]  is  in  (1,  1+1)  and  B[j]  is  in  ( j , j+1))  = 0.5hA(i)hB(j) , Therefore, 

i+j-k  i+j=k-l 

hC*(N)  ^ I I 0.5hA(i)hB(j)  + I I 0.5hA(l)hB(j) 

1 j 1 j 

A.  A Test  Case 

A number  of  normal  random  variables  are  added  using  the  approxi- 
mate convolution  procedure  described  above  and  the  errors  involved  are 
studied. 

Let  N1  be  distributed  as  N(o,l),  l.e.,  normally  distributed  with 
mean  zero  and  variance  1,  and  N1  =•  ^ Nl.  Note  that 

Ni  is  distributed  as  N(0,1). 

Let  Ni'  be  the  histogram  obtained  from  Ni  using  a certain  node 

set. 

Let  NI*  = N1',N2*  = NI*  +'  NI*,  N3*  = NI*  +'  N2*,  NA*  = NI*  +'  N3*,N8* 
NA*  +'  NA*  and  N16*  = N8*  +*  N8*,  where  +'  represents  the  approximate 
convolution  procedure  described  before.  Note  that,  effectively,  Ni* 
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Is  obtained  by  applying  the  procedure  1 times  to  Nl*. 

We  will  try  to  measure  the  error  Involved  In  making  simple 
probability  statements,  l.e.,  those  In  the  form  of  Pr(x  Is  In  I)  • a, 
where  I Is  an  Interval  or  a union  of  Intervals. 

Define 

ERRl(Nj')  - MAX  I |f„.(x)  - f„.,(x)|dx, 

where  1[1]  Is  the  Interval  starting  from  the  1-th  node  of  the  histo- 
gram Nj  ending  at  the  (l+l)-th  node,  and  f^^^  is  the  density  function 
of  Nj , and  f , is  the  density  function  defined  by  the  histogram  Nj ' . 

Define  ERR(Nj')  = I ERRl(NI’)  so  that  ERR(Nj»)  Is  an  upper 
1 

bound  of  error  Involved  in  making  any  simple  probability  statement 
using  f jjj , as  an  approximate  to  f . 

Similarly,  define 


ERRi(Nj*) 


MAX 

l£l[i] 


xel 


and  ERR(Nj*)  = I ERRi(Nj*). 

1 

Since  Nj*  is  an  approximation  to  Nj ' , ERR(Nj*)  Is  an  upper 
bound  to  ERR(Nj*).  Err(Nj*)  - ERR(Nj')  represents  the  error  intro- 
duced by  the  approximate  convolution  procedure. 

The  following  table  shows  the  numerical  results  for  the  case 
where  the  interval  size  is  kept  constant. 
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0.55 
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! 


0.72 


! 16  ! 3.69  ! 2.49  ! 1.47 

•This  N4*  was  obtained  frox  N2*  ♦'  N2* 

TABLE  C"2  Calculated  errors  for  a test  case 
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It  can  be  seen  that  the  error  Introduced  by  the  approximate 
convolution  procedure  Increases  linearly  with  the  effective  number 
of  times  the  procedure  Is  applied.  The  slope  Is  about  0.00009. 

This  Is  practical. 

If  the  Interval  size  Is  Increased  In  the  procedure  (In  order 
to  reduce  the  number  of  Intervals) , then  larger  errors  will  be 
Introduced.  By  keeping  the  Interval  size  constant,  the  number  of 
Intervals  Increases  linearly  with  1,  even  after  eliminating  Intervals 
with  very  small  weights.  This  Is  due  to  flatter  density  functions. 

5.  Discussion 

The  procedure  using  histograms  Is  feasible  and  flexible  for 
adding  random  variables.  In  practice,  It  may  be  necessary  to 
control  either  the  Interval  size  or  the  number  of  Intervals  of  the 
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approximate  histograms.  The  practicality  of  non-«quldlstant  node 
sets  needs  to  be  Investigated. 

For  other  transformations  such  as  multiplication,  the  general 
approach  can  be  extended,  but  it  is  probably  not  as  simple  and 
straightforward . 
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Appendix  III 


THE  MOMENTS  METHOD  FOR  RANDOM  VARIABLE  TRANSFORMATIONS 


1.  Notations: 

The  following  notations  will  be  used  throughout  this  appendix. 
X[l],  1 » l,2,,..,n:  Independent  random  variables 

Yfl],  1 = l,2,.,,,n:  independent  Identically-distributed  random 

variables 

N;  a discrete  random  variable  (Independent  of  all 

Xtl]  and  Y[l],  1 = 1,2,. ..,n.) 

p[l],  1 = 1,2,..,  ; p[l]  Is  the  probability  that  N = 1, 


y^Ci)  = yCl)  or  y(l): 
^ XCJ)  YU) 


Cd.j),  l_j 
E(X) 


the  i-th  moment  of  XtJ]  or  Y[j],  depending  on 
the  context,  with  yQ[j]  being  defined  to  be  1 
for  all  J . 

equals  (1!  )/(j  1 ) ((l-j  ) ! ) . 

the  expectation  of  a random  variable  X. 


2.  Sum  of  Two  Random  Variables 

The  i-th  moment  of  Ys  = Xfl]  + X[2]  is  Jis  = 

E(  (X[l]  + Xt2])^)  = I C(i,k)E(X[l]^)E(X[2]^"S, 

k=0 


written  as 

1 

ys^  = I C(l,k)yj^[l]*y^_^[2]. 


E(Ys^)  = 

which  may  be  re- 


in particular. 
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ysi  - vij^Ii]  + 

^®2  “ ^2^^^  2yj[i]y]^t2]  + V2I2], 

U83  - y3[ll  + 3yj^[lly2[2]  + 3^2111^1121  + U3[2],  and 

ys^  - y^dl  + Ayj^[l]y3[2]  + 6y2(lly2t21  + 4y3[l]y^l2l  + y^[2]. 


3.  Product  of  Two  Random  Variables 

The  i-th  moment  of  Yp  = X[1]X[2]  is  * E(Yp^)  * 
E((X[11X[2])S  •=  E(X[1]^)E(X{2]^).  Therefore,  pp^  = y^[l]  • y^[2]. 


4.  Discrete  Mixture  of  Random  Variables 

The  i-th  moment  of  Yc  = (p[j]:  X[j]  ; j = l,...,n)  is  yc^  >=  E(Yc  ). 

n £ 

Since  Yc^  = (p[j]:  X[j]^;  j =■  l,...,n),  yc.  = ^ p[j]  E(X[J]  ). 

n 3=1 

Therefore  yc.  = X ptj]y^[j]*  Note  that  the  notation  for  a discrete 

^ j=l 

mixture  follows  that  given  in  Appendix  I . 


5.  Sum  of  N Identical  Random  Variables 

To  derive  the  i-th  moment  of  Yr  = 
k 

that  Yr  = (p[k]  : ^ Yj;  k = 1,2,...  ). 

j=l 


of  Yr, 

yri 


00 

I p[k]E{(I  Y )!} 
k=l  j=l  ^ 


n 

^ Yj , we  first  recognize 

j=l 

Therefore,  the  i-th  moment 


Let  y.v  and  Pv  be  the  i-th  moment  of  N and  Y , for  any  j , 

h Yf  1 

The  following  are  the  formulas  for  the  first  four  moment  of  Yr. 


respectively . 
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yii  - 

yr2  ■=  \i\  ]X'l2  ^ 

yr^  - (yN^  - ByN^  + 2yN2)yY^  + SCyN^  - yNj^)yY^  yY^  + yNj^  yY^, 

= (ViN^  - 6yN3  + llyN^  - 6yN3)yYj  + eCyN^  - SyN^  + 2yN^)yYj  yY2 

+ A(yN2  - yNj^)yY3  yY^  + 3(yN2  - yNj^)yY2  + yN^  yY^ 


6.  Continuous  Uniform  Random  Variable' 

The  i-th  moment  of  the  continuous  random  variable  uniformly  dis- 
tributed over  the  Interval  (0,1),  denoted  by  U(0,1),  is  given  by 
l/(i+l).  Specifically  - i , yU2  = y,  yU^  = ^ and  yU^  = y 

7.  Discrete  Uniform  Random  Variable 

Let  Ud(l+n)  be  a discrete  random  variable  distributed  uniformly 

over  1,2,  ...,  n,  then  its  first  four  moments  are: 
n 

yUd  = I (1/n)  i = (n+l)/2, 

1=1 

n 2 

yUd  = I (1/n)  i = (n+l)(2n+l)  /6 
i=l 


yUd_  = ^ (1/n)  i^  = n(n+l) (n+l)/4,  and 

i=l 


y / Q o 

yUd  = I (1/n)  1 = (6n  + 15n  + lOn  -l)/30. 

i=l 


8.  Discrete  Uniform  with  Random  Upper  Bound 

Let  Ud'(l,N)  be  a discrete  random  variable  distributed  uniformly 
over  values  1,2,...,N,  where  N is  a random  variable  or,  more  precisely 
Ud'd.N)  - (p[i]:  Ud(l,i),  i = 1,2,...,  ),  where  pti]  = Pr(N=i). 

The  first  four  moments  of  Ud’(l,N)  are: 

yUdj  = I p[n](n+l)/2  = 0.5(yN^  + 1), 
n 

yUd^  = I p[n](n+l)(2n+l)/6  = (2yN2  + 3yN^  + l)/6, 
n 

yUd^  = I p[n]  (n/4)  (n+1)  = 0.25(yN2  + 2yN2  + yN^^) , and 

n 

yUd^  = I p[n](6n^  + 15n^  + lOn^  - l)/30  = 
n 

(6iiN^  + 15yN^  + lOyN^  - l)/30 


I , 

L 
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FILE  ORGANIZATION  CONCEPTS 

Yuan  Liu 
Amrit  L.  Goel 

Syracuse  University 


Abstract 

In  this  report  we  develop  a rigorous  framework  for  describing 
various  file  organizations  and  operations  on  files.  Various  terms 
pertinent  to  the  description  of  file  organizations  are  defined  and 
a description  of  sequential,  index  sequential,  and  hash  files  is 
presented.  Various  operations  on  these  files  are  also  discussed. 
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FILE  ORGANIZATION  CONCEPTS 

In  this  report  we  first  rigorously  define  some  terms  pertinent 
to  the  description  of  file  organizations.  Then  we  present  a brief 
description  of  sequential.  Index  sequential  and  hash  files  and  de- 
scribe various  operations  on  these  files. 

1.  BASIC  CONCEPTS 

A file  Is  a collection  of  similar  records;  a record  Is  a colleo- 
tlon  of  field  values;  and  a field  value  Is  a unit  of  data  under  con- 
sideration. These  and  other  pertinent  terms  are  rigorously  defined 
here  using  set-theoretic  concepts.  The  following  notations  are  used 
for  this  purpose. 

X «>  cartesian  product, 

• > an  abbreviation  for  a set  operation  used  to  define 

a variable  format  record,  to  be  defined  later, 
u “ union, 

<e>  ® empty  set, 

{ } ® set  definition  by  enumeration,  and 
C ) ■=  ordered  tuple  or  precedence  In  set  definition. 

1,1  Fields  and  Values 

Consider  a number  of  sets;  these  sets  are  called  fields. 
All  other  concept;s  are  derived  from  fields  using  two  set  operations, 
which  are  union  and  cartesian  product.  A field  value  or  value 
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Is  defined  as  a member  of  a field.  A field  value  Is  a basic,  Indl- 
vlsable  unit  of  data  under  consideration.  For  a given  problem,  a 
field  In  one  formulation  may  be  considered  as  an  entity  defined  in 
terms  of  different  fields  In  another  formulation,  so  that  the  field 

value  In  the  former  formulation  appears  to  be  divisible. 

1.2  Formats 

The  terms  "format  J' “fixed  format"  and"varlable  format"are  de- 
fined recursively  as  follows.  A format  is  either  a fixed  format  or 
a variable  format.  A fixed  format  is  a set  which  is  either  a field 
or  a cartesian  product  of  a finite  number  of  other  formats.  A 
variable  format  Is  defined  In  terms  of  a field,  C,  the  control  field, 
a set  of  formats,  S,  and  a one-to-one  correspondence,  M,  between  C 
and  S.  The  set  S may  be  finite  or  Infinite. 

Consider  the  singleton  sets  {c^^},  {C2} where  each  c^^  E C, 

and  formats  S^,  S^,  •••  where  each  c S and  Is  determined  by  the 
correspondence  M(cj^) . A variable  format  Is  defined  as  the  union  of 
the  cartesian  products  of  these  singleton  sets  with  their  corresponding 
formats  In  S.  Thus,  the  definition  of  a variable  format  is  of  the 
form 

({ci)  X Sj^)  u ({cj^}  X $2)0  ... 

This  definition  of  a variable  format  may  be  abbreviated  as  C ® S or 
C ® {Sj^,  S^,  ..•  },  with  an  implied  one-to-one  correspondence  between 

C and  S. 
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Although  a format  may  be  defined  in  terms  of  other  fotmats, 
every  format,  after  a finite  number  of  substitutions,  must  be  ulti- 
mately defined  in  terms  of  fields. 

IVo  formats  are  equivalent  if  they  can  be  made  identical  by 
applying  the  following  rules  to  their  definitions: 

(1)  A X B - B X A 
(il)  A X (B  X C)  •=  (A  X B)  X C 

(iii)  A X (B  u C)  - (A  X B)  u (A  X C) 

where  A,  B and  C are  formats.  This  concept  is  useful  when  fields 
in  a format  are  rearranged  to  form  a key  (to  be  defined)  while 
maintaining  an  equivalent  format. 

1.3  Records  and  Logical  Files 

A record  is  defined  to  be  a member  of  a format;  a fixed  format 

record  is  defined  to  be  a member  of  a fixed  format,  and  a variable 

format  record  is  defined  to  be  a member  of  a variable  format.  In  a 
variable  format  record,  the  value  of  the  control  field  indicates  the 
format  of  the  rest  of  the  record. 

A logical  file  or  a file  is  defined  to  be  an  empty  set  or  a 
collection  of  records.  Equivalently,  a file  may  be  defined  to  be  a 
subset  of  a format.  This  definition  does  not  allow  two  identical 


records  in  a file  due  to  the  nature  of  a set.  From  this  viewpoint, 

I 


two  Identical  physical  records  stored  in  a computer  system  are  not 
truly  identical,  because  each  record  has  an  implicit  field,  its 
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1,4  Keys 


A primary  key  is  a field  or  a format  defined  from  a collection 
of  fields  of  a given  format  and  the  value  or  values  of  the  primary 
key  uniquely  Identify  each  record  In  any  file  that  Is  considered  legi- 
timate. That  Is,  a primary  key  may  exclude  certain  files  or  a format 
from  consideration.  A key  Is  a field  or  a format  defined  from  a col- 
lection of  fields  of  a given  format.  The  word  "key"  connotes  the 
usage  In  defining  a record  or  a collection  of  records  In  a file  or 
a format  by  giving  conditlon(s)  on  the  field  value(s)  of  the  key. 

If  K is  a format  consisting  of  only  the  fields  in  a key,  then  there 
exists  a format  L such  that  K x L is  equivalent  to  any  format  that 
contains  all  the  fields  of  the  key. 

1.5  Physical  Files  and  File  Organizations 

A physical  file  Is  a representation  of  a logical  file  on  a 
storage  medium.  A file  organization  Is  a physical  file  or  a collec- 
tion of  physical  fiels  used  to  represent  a logical  file.  In  the 
representation  process,  each  field  value  and  record  of  the  physical 
file  Is  assigned  a physical  address  and  every  record  of  the  logical 
file  corresponds  to  a record  of  the  physical  file. 

A physical  file  may  have  a different  format  from  that  of  the 
represented  logical  file,  and  a file  organization  may  contain  additional 
files  to  facilitate  operations  against  the  file.  For  example,  physical 
address  and  record  length  (in  bytes,  words,  etc.)  may  be  introduced  as 
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additional  fields;  a field  whose  values  are  variable-length  nay  be 
redefined  as  the  cartesian  product  of  several  fixed-length  fields; 
an  index  file  may  be  introduced  as  an  additional  file,  as  in  index 
sequential  files. 

1.6  Operations  on  File  Organization 

The  basic  operations  on  a file  organization  are  read,  add, 
delete,  and  update,  which  involve  single  records.  All  other  file 
operations  can  be  described  in  terns  of  these  basic  operations. 

The  read  operation  makes  available  a particular  record  which 
satisfies  a condition  or  determines  whether  such  a record  exists. 

In  case  a condition  specifies  more  than  one  record  in  the  file  organi- 
zation, all  such  records  are  considered  to  form  a file  with  a certain 
ordering.  We  read  the  first  record  that  satisfies  the  condition  and 
then  repeatedly  read  the  next  record  vmtil  the  read  operation  indi- 
cates no  more  records. 

The  add  operation  adds  a given  record  to  the  file  organization, 
possibly  at  a specific  relative  position. 

The  delete  and  update  operations  operate  on  a record  which  has 
just  been  read.  If  the  operations  are  to  be  done  on  multiple  records 
specified  by  a certain  condition,  a number  of  read  operations,  e.g., 
read  first  on  the  condition  and  read  next  on  the  condition,  may  be 
carried  out  before  the  delete  or  update  operation.  To  delete  a 
record  means  either  to  remove  it  from  the  file  organization  or  to  change 

it  to  a special  record,  a deleted  record,  that  does  not  satisfy  any 
conditions  on  the  records. 
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2.  EXAMPLES 


A few  examples  are  now  given  to  Illustrate  the  above  definitions 
and  concepts.  The  following  fields  are  used  in  these  examples. 

I “ the  set  of  non-negative  Integers, 

NAME  > the  set  of  all  personal  names, 

A = the  set  of  alphanumerical  characters, 

ADDRESS  = the  set  of  all  addresses,  and 

CAR  = the  set  of  all  car  makes. 


l.e. 


Example  1 Let  El  be  the  cartesian  product  of  NAME  and  ADDRESS, 


El  = NAME  * ADDRESS 


Note  that  El  is  a fixed  format. 

Let  John  Smith  and  Mary  Dean  be  two  Instances  of  values  of  the 
field  NAME.  Similarly,  let  113  Webster  Street,  156A0  Salina  Street, 
and  3364  Main  Ave.,  Apt.  3B  be  three  instances  of  the  field  ADDRESS. 
Then  the  ordered  pair  (John  Smith,  113  Webster  Street)  is  an  exaiqjle 
of  a-record  of  the  format  El.  The  set 


{(John  Smith,  113  Webster  St.), 
(Alexander  Moore,  15640  Salina  St.), 
(Mary  Dean,  3364  Main  Ave.,  Apt.  3B)} 
is  an  example  of  a file  of  format  El. 
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Example  2 Let  format  E2  be  given  by 

E2  ^ 1 9 CC^e^^f  CAE^f  CAR^ 0 CAR^y 

where  CAR-  denotes  CAR  and  CAR  denotes  the  n-fold  cartesian  product 


of  CAR.  Note  that  E2  Is  a variable  format  and  I is  the  control 
field.  Each  value  from  I in  a record  indicates  the  format  of  the 
record,  i.e.,  the  "n"  of  CAR^. 

Examples  of  records  in  this  format  are: 

( 0,  <e>  ) 

( 1,  Dodge) 

(4,  (Ford,  Ford,  Dodge,  American)  ) 


Example  3 Let  E3  be  given  by 

E3  = NAME  X (C  ® {{<e>}  , CAR}) 

E3  is  another  example  of  a variable  format  where  the  control 
field  C has  exactly  two  members  each  indicating  whether  {<e>}  or  CAR 
is  the  relevant  format  for  the  rest  of  a record  in  E3.  The  follovring 
is  a file  in  E3. 

{(John  Smith,  (0,  <e>)), 

(Mary  Dean,  (1,  Ford)), 

(Alexander  Moore,  (1,  Dodge))} 

The  field  CAR  in  format  E3  is  sometimes  called  an  optional  field, 
since,  in  a sense,  it  may  or  may  not  exist  for  a record  in  E3. 


Example  4 Let 


E4  = NAME  X I ® {Sp,  S^,  S^, 

where  Is  a subset  of  CAR  with  i metribers  and  a member  1 of  I corre- 
sponds to  S^,  for  i ■>  0,  1,  2^  ... 

This  Is  another  example  of  a variable  format.  Since  a member 
of  is  a file  of  the  format  CAR,  each  record  of  EA  contains  a file 
of  CAR  which  Is  called  a subsumed  file  of  the  record.  The  value  of 
the  control  field  I Indicates  the  ntnnber  of  records  In  the  subsumed 
file. 


3.  SEQUENTIAL  AND  DIRECT  FILES 
3.1  Sequential  Files 

A sequential  file  is  a physical  file  In  which  one  record  appears 
"after"  another  In  some  sense  and  this  order  Is  determined  by  the 
physical  addresses  of  the  records.  Note  that  every  sequential  file, 
unless  empty,  has  exactly  a first  record  and  a last  record. 

A physical  sequential  file  is  a sequential  file  in  which  a record 
A follows  a record  B if  and  only  If  A Is  stored  after  B,  in  the  sense 
that  no  valid  data  Is  stored  "between"  A and  B and  the  address  of  A is 
"greater"  than  B as  determined  by  the  characteristics  of  the  storage 
medium. 

A pointer  sequential  file  is  a sequential  file  whose  ordering 
is  represented  by  an  added  pointer  field.  As  an  example,  consider  a 
pointer  sequential  file  of  the  format  S = F x R,  where  R is  the  format 
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of  the  represented  logical  file  and  F Is  the  set  of  all  record 

addresses.  In  this  example,  the  value  of  F In  a record  of  S Indl-  i 

cates  the  address  where  the  next  record  Is  assigned.  The  value  of 

F In  the  last  record  Identifies  the  record  as  the  last  one  In  the 

file. 

A variation  of  the  above  pointer  sequential  file  has  the  format  ] 

I 

j 

F X R , where  R Is  the  n-fold  cartesian  product  of  R.  Yet  another  ; j 

n n ] I 

! J 

variation  has  the  formal;  F x B x R,  where  B Is  the  backward  pointer  ■ ; 

i i 

field  that  contains  the  physical  address  of  the  previous  record.  The  j i 

!i 

value  of  B In  the  first  record  Indicates  the  record  as  the  first.  . I 

Any  physical  file  may  be  considered  to  be  a sequential  file,  as 
every  record  has  a physical  address  which  determines  the  next  and  '■ 

prior  records . | 

In  a sequential  file.  If  record  A folows  record  B,  then  the  j 

fi 

address  of  A Is  not  known  until  B Is  examined  either  for  Its  pointer  | 

field  (pointer-sequential)  or  for  the  address  of  record  B (physical 
sequential) . 


aerations  on  Sequential  Files 


The  basic  read  operations  are  read  the  first  record,  and  read 
the  next  record  according  to  the  orderii\g  of  the  sequential  file. 

Certain  sequential  files  also  allow  the  operations  read  the  last 
record  and  read  the  prior  record.  To  read  a record  satisfying  a 
condition,  the  records  from  the  first  record  on  must  be  read  one 
by  one  In  the  order  specified  by  the  sequential  file,  or,  alternatively. 
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the  records  are  read  from  the  last  record  backward  If  possible. 


It  may  not  be  possible  to  add  a record  to  a physical  sequential 
file  If  the  added  record  must  be  between  two  specific  records  and 
there  Is  no  space  between  them.  In  this  case  a new  physical  file 
must  be  generated  or  the  add  operation  falls.  In  a pointer  sequen- 
tial file,  a new  record  Is  added  between  any  two  records  by  adjusting 
the  pointers. 

3. 3 Ordered  Sequential  Files 

It  Is  possible  to  maintain  a sequential  file  such  that  the  next- 
prior  relation  of  the  records  coincides  with  a condition  on  the  values 
of  a key.  Such  sequential  flies  are  said  to  be  ordered.  For  Instance, 
the  value  of  a certain  field  K In  a record  A may  be  kept  to  be  always 
smaller  than  or  equal  to  the  value  of  K In  the  next  record  of  A (records 
In  ascending  sequence) . When  add,  delete  and  update  operations  are 
performed  on  ordered  sequential  files,  care  must  be  taken  to  preserve 
the  ordering  condition. 

3.4  Direct  Files 

A direct  file  is  a physical  file  stored  in  such  a way  that  (1)  the 
range  of  all  physical  addresses  Is  known  without  accessing  any  physical 
file,  and  (ii)  given  the  address,  each  record  may  be  found  directly, 
tlie  latter  is  a requirement  on  the  storage  medium. 

Dfrt'  t files  are  closely  related  to  sequential  files.  As  noted 

I I file  m.iy  be  used  as  a sequential  file,  since  the  first 
' for  »arh  record  arc  defined  by  the  addresses 


1 


of  the  records.  In  fact,  a direct  file  Is  also  a physical  sequen-  | 

tlal  file  ordered  by  the  field  physical  address  (Implicit  or  explicit) . 

j 

Reversely,  sequential  files  may  be  used  as  direct  files  If  J 

(1)  the  set  of  all  possible  addresses  Is  known,  (11)  unused  space 

j I 

always  stores  the  representation  of  a deleted  record,  and  (111)  the  I | 

storage  medium  permits  random  access. 

In  a direct  file,  read  Is  simple.  To  delete  a record,  either 
It  Is  marked  as  deleted  or  Its  address  Is  removed  from  the  set  of 
possible  addresses.  The  add  and  update  operations  are  straight  for- 
ward. 

3.5  More  Complicated  File  Organizations 

In  a more  complicated  file  organization,  the  records  are  par- 
titioned and  each  partition  Is  stored  as  a sequential  file,  a direct 
file  or  another  file  organization. 

For  example,  consider  the  case  when  a record  A must  be  added 
between  two  records  B and  C In  an  ordered  physical  sequential  file,  ' 

but  there  Is  no  space  between  B and  C.  In  order  to  avoid  the  re- 
generation of  the  entire  file  In  such  a situation,  an  overflow  tech- 
nique may  be  used.  With  such  a technique,  the  records  are  partitioned 
Into  a number  of  subfiles,  which  are  stored  as  pointer-sequential  files. 

The  first  buckets  of  these  subfiles  are  stored  as  physical  sequential  j 

files  In  one  area  and  the  rest  are  stored  in  a separate  area. 

j 
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4.  INDEX  SEQUENTIAL  FILES 
4.1  Description  of  the  File 

An  Index  sequential  file  consists  of  a data  file  and  a hierarchy 
of  Index  files.  Each  record  In  the  data  file  corresponds  to  a record 
in  the  logical  file  uhlch  the  Index  sequential  file  represents.  The 
Index  files  are  supplementary  files  and  are  used  to  facilitate  opera- 
tions on  the  file. 

To  use  the  index  sequential  file  organization  technique,  a key  must 
be  defined. 

The  data  file  may  use  any  file  organization  technique  and  is  partitioned 
Into  a number  of  data  subfiles.  An  example  Is  to  store  the  data  file 

as  a physical  sequentail  file  of  the  subfiles,  where  each  data  sub- 
file has  a fixed  number  of  records.  Another  example  Is  to  store  each 
subfile  as  a pointer-sequential  file  and  to  store  the  first  buckets 
of  the  data  subfiles  as  a physical  sequential  file,  while  keeping 
the  records  ordered  by  the  key. 

An  Index  file  for  a file  of  format  K x F is  of  format  R x A, 
where  R is  the  set  of  ranges  of  K and  A is  the  set  of  physical  addresses 
pointing  at  subfiles  of  the  Indexed  file.  The  key  range  field  R of 
an  index  record  contains  the  range  of  the  key  values  of  the  records 
In  the  subfile  pointed  at  by  the  pointer  field  A. 

An  index  file  I may  be  created  and  maintained  for  a data  file 
or  for  another  Index  file  J.  In  the  latter  case  the  R field  of  I 
contains  the  union  of  ranges  of  the  records  in  a subfile  of  J.  In 
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this  way,  an  Index  sequential  file  stay  have  a hierarchy  of  Index 
files.  We  will  say  that  the  Index  file  for  the  data  file  Is  at  level 
li  the  Index  file  for  the  Index  file  at  level  1 Is  at  level  2,  and  so 
on.  The  Index  fll^  ah  the  greatest*nuiid>ered  level  Is  referred  to  as 
the  top-level  Index. 

The  Index  subfiles  are  all  ordered  sequential  flies  and  are 
usually  physical  sequential  files. 

4.2.  Operations  on  the  File  | 

To  read  a record  satisfying  a condition  on  the  key.  the  top-  i 

level  Index  Is  serially  searched  to  locate  the  siibflle  of  the  lower- 
level  Index  whose  key  value  range  contains  a key  value  satisfying  the 
condition.  The  same  search  Is  done  at  the  still-lower  level  Index 

j 

and  repeated  until  a data  subfile  Is  located.  The  data  subfile  Is 
then  searched  serially  to  read  a record  satisfying  the  given  condition. 

To  add  a record,  the  record  Is  Inserted  In  the  data  subfile  as  : 

determined  by  the  Index  files.  If  the  nundier  of  records  In  a data 
subfile  Is  fixed,  It  may  become  necessary  to  create  a new  data  sub- 
file and  hence  cause  the  adding  of  a record  Into  the  Index  file  at  ^ 

level  1,  which  may  In  turn  cause  the  creation  of  a new  index  subfile 
and  the  addition  of  a record  In  the  index  file  at  level  2.  This 
rippling  effect  may  continue  to  the  top-level  Index  and  finally  cause 
an  extra  level  of  index  file  to  be  created. 

To  delete  a record  means  to  delete  the  record  from  the  data  file. 

This  Is  done  according  to  the  flic  organization  used  for  the  data  file.  I 

i 

1 

i 
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If  a data  subfile  becomes  empty.  It  may  be  desirable  to  delete  the 
corresponding  Index  record  at  level  1,  which  may  In  turn  cause  a 
subfile  at  level  1 to  become  empty.  Similar  to  adding  a record, 
this  rippling  effect  may  extend  to  the  top  level  and  may  cause  the 
top  level  to  disappear. 

To  update  a record,  If  the  key  value  is  not  changed,  the  record 
Is  read  and  updated.  If  the  key  value  Is  changed,  the  original  record 
Is  first  deleted  and  then  a new  record  reflecting  the  desired  changes 
Is  added. 

5.  HASH  FILES 

5.1  Description  of  the  File 

A hash  file  has  a hashing  function  that  determines  an  address, 
the  home  address  for  any  given  key  value  and  a key  must  be  defined.  ! 

All  records  that  have  the  same  home  address  form  a collision  set. 

TVo  records  belonging  to  the  same  collision  set  may  either  contain 
Identical  key  values  or  different  key  values  that  correspond  to  the  I 

same  home  address.  | 

A hash  file  has  a primary  area  and  an  overflow  area.  The  latter  ■ 

may  be  a separate  area  of  may  coincide  with  the  primary  area.  The 
home  addresses  generated  by  the  hashing  function  point  at  buckets,  ! 

the  home  buckets  In  the  primary  area.  Each  home  bucket  can  only 

store  a fixed  number  of  records.  Tliose  records  that  cannot  be  stored  i 

1 

In  their  home  buckets  are  called  overflow  records  and  are  stored  in  | 

the  overflow  area.  I 
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A hashing  algorithm  detezmlnes  a series  of  addresses  In  the 
overflow  area  based  on  a given  key  value.  An  overflow  record  Is 
stored  at  one  of  these  overflow  addresses.  Note  that  a record  must 
be  stored  either  at  the  home  address  or  at  one  of  the  overflow  addresses 
and  that  all  these  addresses  may  be  predetermined  from  the  key  value 
of  the  record  by  the  hashing  function  and  the  hashing  algorithm. 

To  read  a record,  the  direct  file  composed  of  buckets  at  this 
predetermined  series  of  addresses  Is  searched. 

If  a separate  overflow  area  Is  used,  the  overflow  buckets  for 
the  same  collision  set  may  be  linked  together  as  a pointer  sequential 
file.  In  this  case,  the  hashing  algorithm  Is  used  only  to  allocate 
buckets  In  the  overflow  area. 

If  the  overflow  area  coincides  with  the  primary  area  and  it  Is 
desired  to  use  pointer  sequential  file  organization,  there  must  be  a 
pointer  for  each  record  to  link  together  all  the  records  in  the  same 
collision  set. 

All  hash  files  may  be  derived  by  varying  the  hashing  function, 
the  hashing  algorithm,  and  the  way  the  overflow  records  are  stored 
and  retrieved.  For  some  examples  of  various  hash  files, (see  Knuth 
0-973),  Buchholtz  0.963) » Morris  0.968 )and  Peterson  (l957)  • 

The  hash  file  organization  may  be  used  for  an  index  file,  leaving 
the  data  file  free  to  use  any  file  organization. 

10-15 


5. 2 Operations  on  the  File 


The  hash  file  must  be  Initialized  to  fill  the  primary  area  with 
deleted  records  before  any  operation  can  be  performed  on  it. 

To  read  a record  with  a given  key,  the  buckets  at  the  home 
address  and  overflow  addresses  are  searched.  Note  that,  if  allowed 
by  the  storage  device,  it  is  possible  to  design  the  home  and  overflow 
addresses  such  that  all  the  home  and  overflow  buckets  are  searched 
simultaneously.  To  read  a record  satisfying  an  arbitrary  condition 
requires  the  records  to  be  read  one  by  one,  treating  the  physical 
file  as  a sequential  file. 

To  add  a record,  we  search  for  a deleted  record  among  the 
buckets  at  the  home  address  and  the  overflow  addresses.  If  one  is 
found,  the  record  is  stored  there,  possibly  updating  the  pointers 
in  a pointer  sequential  file;  othervjise  the  add  operation  fails. 

There  are  various  ways  to  limit  the  search  for  a non-existent  record. 

To  delete  a record,  the  found  record  is  simply  changed  to  a 
deleted  record. 

To  update  a record,  the  foxmd  record  is  simply  changed  as 
desired  unless  the  key  value  is  changed.  In  the  latter  case,  the 
old  record  must  be  deleted  and  the  new  record  added. 
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Section  11 

CONCURRENCY  IN  HASHED  FILE  ACCESS 


Leo  H.  Groner 
Amrit  L.  Goel 

Syracuse  University 

Abstract 

This  report  presents  a technique.  Concurrent  Hashed  Overflow 
(CHO) , for  improving  the  performance  of  hashed  file  systems  in 
terms  of  both  access  counts  and  storage  efficiency.  The  technique 
involves  the  exploitation  of  the  ability  of  many  existing  computer 
systems  to  access  more  than  one  unit  of  data  concurrently.  Algorithms 
implementing  CHO  are  presented  and  an  analytic  model  describing 
its  performance  is  developed.  The  problem  of  optimizing  the 
performance  of  a CHO  file  is  formulated  as  a dynamic  programming 
problem  and  is  solved  numerically.  Tables  are  presented  giving 
dynamic  programming  optimum  stage  allocations  and  next  stage  state. 
From  these  tables  optimum  CHO  designs  may  be  easily  obtained. 
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1.  INTRODUCTION 


In  the  design  of  hash  coded  files  it  is  generally 
accepted  that  the  trade-off  between  performance  efficiency 
and  storage  efficiency  is  inevitable.  This  trade-off  is 
investigated  by  Van  der  Pool  [11,  12,  13] . However,  there 
exists  a third  variable,  moreover  one  that  is  frequently 
vinderutilized,  that  can  be  traded  for  improvements  in  both 
performance  and  storage  utilization.  This  variable  is  the 
level  of  concurrent  device  utilization.  In  this  report  we 
try  to  exploit  the  ability  of  many  existing  computer  systems 
to  access  data  on  more  than  one  device  concurrently.  We 
present  cin  algorithm  which  we  call  concurrent  Hashed  Overflow 
(CHO)  which  may  be  used  to  yield  improvement  in  hashed  file 
systems  in  terms  of  both  access  time  and  storage  efficiency. 

To  access  a record  using  CHO,  an  address  set  consisting 
of  the  initial  address  and  overflow  addresses  is  generated 
by  a set  of  hashing  functions.  When  this  set  of  addresses 
represents  distinct  devices,  seeks  to  these  addresses  may 
be  performed  concurrently.  Since  more  thain  one  bucket  is 
retrieved,  the  probability  of  success  in  obtaining  the  desired 
record  in  one  access  cycle  is  increased.  Algorithms  implement- 
ing CHO  are  presented  and  an  analytic  model  of  its  performance 
is  developed.  Allocating  a CHO  file  evenly  between  devices 
may  not  be  an  optimal  policy.  The  problem  of  CHO  file  design 
for  optimum  performance  is  formulated  as  a dynamic  programming 
problem  of  the  form;  determine  storage  allocations  to  minimize 
expected  accesses  for  a given  level  of  concurrency,  load 
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factor,  bucket  size  and  file  size.  This  problem  is  solved 
numerically  and  tables  are  presented  giving  a dynamic  pro- 
gramming problem  optimum  state  allocations  and  next  stage 
state.  These  tables  may  be  used  in  optimum  CHO  file  design. 

2.  CONCURRENT  HASHED  OVERFLOW  PROCEDURE 
2.1  General  Description 

Consider  a hash  coded  file  consisting  of  a primary 
area  into  which  records  are  hashed.  Collision  overflows  from 
this  area  can  be  handled  in  a number  of  ways.  Chaining  and 
open  addressing  are  among  the  more  popular  techniques.  One 
technique  that  might  be  considered  involves  the  use  of  a 
separate  overflow  file  F2  for  overflows  from  the  primary 
file  Fj^  . F2  may  itself  be  organized  as  a hash  coded  file. 
Overflows  from  this  file  may  be  stored  in  a third  overflow 
file  F^  • For  each  hash  coded  file  in  the  sequence 
^1'  ^2'  f’3'***'  we  may  associate  a sequence  of  dif- 

ferent hashing  functions  h^  which  map  each  key  k into  a 
sequence  of  addresses  a^  . This  sequence  represents  all  the 
possible  addresses  where  the  record  may  be  found.  The  important 
point  concerning  this  sequence  is  that,  given  the  key,  the 
entire  sequence  may  be  immediately  calculated. 

If  the  files  are  each  assigned  to  a separate  device, 

l/o  operations  may  be  initiated  immediately  to  each  file. 

For  devices  such  as  the  IBM  2314,  disc  seeks  on  different 
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units  may  be  overlapped.  For  the  IBM  3330*  both  seeks  and 
portions  of  rotation  latency  may  be  overlapped.  The  por- 
tions of  the  file  accesses  which  cannot  be  overlapped  are  the 
start  l/oa  and  the  final  examination  of  record  keys  In  main 
memory.  Sven  transmission  of  records  from  the  devices  to 
main  memoiry  may  be  overlapped  to  the  extent  that  the  devices 
may  be  assigned  to  different  channels.  For  the  2314  and  3330, 
representative  times  In  milliseconds  for  various  operations 
are  given  In  Table  1.  From  this  table  we  see  that  the  bulk 
of  the  time  spent  in  accessing  records  may  be  overlapped. 

Since  the  performance  consequence  of  collision  over- 
flow may  be  reduced  by  concurrent  accesses  to  the  overflow 
chain*  the  possibility  of  decreasing  the  storage  requirements 
by  increasing  the  file  load  factor  presents  Itself.  Although 
the  number  of  file  accesses  may  increase*  the  number  of  file 
accesses  cycles*  l.e.*  non-concurrent  accesses,  is  decreased. 
It  is  the  number  of  access  cycles  that  is  the  primary  deter- 
minant of  response  time. 

2.2  Flowcharts 

The  flowcharts  in  Figs.  1 and  2 demonstrate  the  concepts 
discussed.  In  this  example  four  levels  of  files*  a primary 
and  three  levels  of  overflow*  are  used.  Use  of  the  terms 
FORK,  JOINT  and  TERMINATE  follows  the  terminology  of 
Anderson  til  • 

In  Fig.  1*  the  HASHREAD  routine  sets  a switch  EXISTS 
and  sets  indexes  RI  and  RJ  to  point  to  the  proper  buffer  in 
BUCKET  where  the  sought  record  may  be  fovind.  HASHREAD  forks 
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TABLE  1 

TIMING  FIGURES  FOR  IBM  2314  AND  3330  DASD 


2314 


3330 


Start  l/o 

.030 

.030 

Seek 

25-130 

10-55 

Latency 

0-25 

0-16.7 

Transmit 

0-25 

0-16.7 

Record  Examination 

.050 

.050 
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2.3  A PL/ 1 Implementation 


The  PL/1  progrEun  given  in  Fig.  3 is  em  Implementation 
of  the  flowcharts  in  Figs.  1 and  2. 

The  MAINl  routine  defines  the  environment  and  calls 
the  HASHREAD  routine  which  sets  EXIST  and  sets  RI  and  RJ  to 
point  to  the  proper  buffer  in  the  BUCKET  structure.  HASHREAD 
initiates  four  READBUCKET  svibtasks  which  operate  concurrently. 
If  the  record  is  found,  an  interrupt  is  generated  by  signaling 
DONE,  transferring  control  to  the  OUT  statement  in  HASHREAD. 
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MAINl:  PROCEDURE  OPTIONS (MAIN) ; 

DCL(RI,RJ)  FIXED  BINARY(31),  ARC  CHAR(6); 

DCL  (F1,F2,P3,F4)  FILE  RECORD  DIRECT  ENVIRONMENT  (REGIONAL(l) ) ; 
DCL  1 BUCKET(4),2  RECORD(10)«3  (KEY  CHAR(6),  DATA  CHAR  (24)); 
DCL  EXISTS  BIT(1)EV(4)  EVENTS; 

DCL  1 TARGET_RECORD, 2 (KEY  CHAR (6). DATA  CHAR(24)); 

CALL  HASHREAD (ARG) ; 

IF  EXISTS  THEN  TARGET_RECORD=BUCKET (Rl) . RECORD (RJ) ; 

EISE  /*  WHATEVER  */; 

HASHREAD ; PROC (ARG ) ; 

ON  CONDITION (DONE)  GOTO  OUT; 

EXISTS='0'B; 

DO  1=1  TO  4; 

CALL  READBUCKET(I,ARG)  TASK  EVENT (EV(I) ) ; 

END; 

WAIT(EV(1),EV(2),EV(3),EV(4))  (4);/*WAIT  FOR  ALL  OTHER 

EVENTS  TO  COMPLETE  */ 

OUT; RETURN; 

DCL  1 FIXED  BINARY(31)  R(4)  LABEL; 

DCL  TBUCKET  CHAR (200) .ADDRESS  FIXED  BINARY  (31); 

ADDRESS=HASH ( I , ARG ) ; 

GOTO  R(I) ; 

R(1):READ  FILE (Fl) INTO (TBUCKET  ) KEY (ADDRESS ) ; 

BUCKET ( I ) =TBUCKET ; 

GOTO  TEST; 

R(2);READ  FILE (F2) INTO (TBUCKET  ) KEY (ADDRESS ) ; 

BUCKET ( I ) =TBUCKET ; 

GOTO  TEST; 

R ( 3 ) ; READ  FILE (F3 ) INTO (TBUCKET  ) KEY (ADDRESS ) ; 

B ICKET ( I ) =TBUCKET ; 

GOTO  TEST; 

R(4);READ  FILE  (F4 ) INTO (TBUCKET  ) KEY (ADDRESS ) ; 

BUCKET ( I ) =TBUCKET ; 

TEST: DO  J=1  TO  10; 


I Fig.  3 A PL/1  implementation  (cont. ) 

i 


IF  BUCKET (I). RECORD (J),KEY=ARG 


THEN  DO  R=I»RJ»J;EXISTS=' I'B; 

SIGNAL  CONDITION (DONE) ; 

END 

END;/*  OF  J lOOP  */ 

IF  (I«10)  EXIST_OVERFLOW (BUCKET) (10) ) 

THEN  CALL  OVERFLOW  (ADDRESS,  ARG)  ; 

/♦PUTS  OVERFLOW  ADDRESS  IN  RI,RJ  AND  FILLS  BUCKET  (K)*/ 

END  READBUCKET;  - 

EXIST  OVERFLOW; PROC (BUCKET)  RETURNS (BIT ( 1) ) ; 

/•DUMMY  ROUTINE*/  RETXJRN  ( ' 0 • B ) ; I 

END  EXIST_OVERPLOW; 

OVERFLOW:  PROC  (ADDRESS , ARGUMENT ) ; 

/♦DUMMY  ROUTINE*/ 

END  OVERFLOW; 

HASH; PROC (I, ARG)  RETURNS (FIXED  BINARY(31)); 

DCL  I FIXED  B INARY ( 31) , ARG  CHAR(6) ; 

/*  DUMMY  */  RETURN(IO); 

END  HT^H; 

END  HASHREAD; 

END  MAINl; 

Fig.  3 A Pl/1  implementation. 


3.  ANALYTICAL  MODEL  AND  OPTIMIZATION  METHODOLOGY 


t 


One  way  of  formulating  the  problem  of  optimizing  the 
design  of  Concurrent  Hashed  Overflow  files  is*  minimize  the 
extra  accesses  due  to  the  kth  stage  overflow  subject  to  an 
overall  load  factor  Lq  , bucket  size  s and  a requirement  for 
Tq  = LqSB  records  to  be  hashed,  where  B is  the  number  of 
buckets. 

We  are  given  = sB  storage  positions  for  records, 
i.e.,  slots.  We  allocate  slots  to  device  i,  i = 0, 1,  ,k-l  . 

Following  Van  der  Pool  [11],  the  overflow  records  from 

the  (i+l)th  stage  satisfy; 


~ro- 

1 s 
e m. 


^i+1  ^i 


.^3 


~™i  j 
e m^ 


where  = sr^/x^  . Slots  remaining  from  the  (i+l)th  stage 
satisfy;  a.^i=c.-x  . (See  Fig.  4)  Letting  A.  = r ./x. 

H"  X X X X X A 

and  load  factor  after  stage  (i+1)  is  given  by 


A^L^  -SA^  (sA^) 


- (1-A^)  e 


S..  s-1  (SA.  ) 

j=0 


\ 

i 


The  stage  to  stage  transition  has  now  been  expressed  in  terms 
of  a single  state  variable  L^  . Denoting  the  above  expression 
by  t(L^,A^)  we  have  L^^  = t (L^,  A^)  . (See  Fig.  5.)  We  note 
that  is  the  return  from  stage  i and  is  a function  of 

the  state  variable  L^^  and  decision  variable  A^^  . 

Let  R.(L.,A.)  = L^  1 if  i=k-l  and  0 otherwise, 

X X 1.  iC“  X 


“ nin  for  i.<]C“2  , 

Ai  » L^ 

«k-2<\.2>  “ ^<\-2*\-2J  i=k-2. 

Then  for  i<k-2,  , f^ui<'^<‘-i-'Sc>>l  • 

Ai»Li 

Since  monotonic,  minimizing  the  expression  above 

is  equivalent  to  minimizing  t(L^,Aj^)  over  A^  subject  to 
Aj^  A £ . Using  Lagrange  multipliers  we  wish  to  minimize 

F{Lj^.A^,X^,u^)  = t(L^,A^)  + l^(A^-L^-uJ) 

by  setting: 


-aJL 

aA, 


dt 

dA. 


+ X.,  = 0 


(3) 


= A.  - L.  - Uj  = 0 
8X^  i i i 


(4) 


ik  = - = 0 


(5) 


Eq.  (5)  holds  if  and  only  if  = 0 u^ = 0 . Assuming 

\^  = 0 , Aj^  may  be  determined  by  solving  ® ^ 

solution  At  . Alternatively,  assuming  u^  = 0 implies  A^  = 
Then  A^ = max{At , L^]  . Dropping  the  subscripts  we  get 


dt 

dA 


-SA  (SA)^^J 


V j=0  =>• 


s! 


+ (A-l)[e®^-  (SA)^]]]  = 0. 


(6) 
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Let  A*  be  the  solution  to  equation  (6).  Then  *»  t (L^»AJ) 

is  minimum.  The  algorithm  for  generating  the  optimal  alloca- 
tions is  therefore j 

1.  initialize  Lq  , set  i*0,  and  set  k equal  to  the  number 
of  stages. 

2.  Compute  optimum  AJ  given  . 

3.  Set  X,i^l.t(L,Al)  . 

4.  !•  1.  1 . 

5.  If  i<k-l  , then  go  to  2 , else  go  to  6 . 

^-l^^-l* 

Given  the  A*,....A*_2^  we  may  use  eq.  (1)  to  determine  the 
number  of  records  in  each  stage,  ^o'**''^k-l  ' slots 

allocated  to  them  *^o****k-l' 


i 1 

i 

. ^ . 

4.  TABLES  FOR  DESIGN 

The  optimum  allocated  load  factor  A^  at  different  bucket 
' sizes  s and  stage  input  load  factors  is  obtained  by 

solving  eg.  (6)  numerically.  These  values  are  given  in  Table 
2 2md  Fig.  6 is  a plot  of  the  data  in  Table  2.  Different 
curves  are  given  for  different  values  of  s . 

Table  3 la  obtained  by  applying  eg.  (2)  for  the  At 
values  of  Table  2.  Fig.  7 is  a plot  of  data  from  this  table  1: 

I 

giving  output  load  factors  when  optimum  allocated  load  factors  i 

are  used  as  a function  of  input  lord  factors.  The  different 
curves  represent  different  bucket  sizes  s . 

* 5.  NUMERICAL  EXAMPLE 

^ The  following  example  may  be  illustrative.  Suppose  we 

are  given  a file  of  100000  records  with  a bucket  size  10  and  i 

1 

an  overall  load  factor  0.90.  A primary  file  and  two  levels  j 

of  overflow  are  to  be  used.  They  will  be  stored  using  the 
CHO  technigue  on  three  separate  devices.  Thus  the  file  will 

I 

be  stored  in  11112  buckets  allocated  as  given  below.  Over-  I 

flow  from  File  3 is  490  records  and  these  may  be  stored  in 

File  3 using  an  OPEN  addressing  scheme.  j 

I * 


TABLE  2 

OPTIMUM  ALLOCATED  LOAD  FACTOR  At  TO  STATE  (i+1)  - 


bucket  size,  s 


L. 

1 

1 

2 

3 

5 

10 

25 

0.200 

0.305 

0.273 

0.256 

0.239 

0.222 

0.215 

0.300 

0.461 

0.415 

0.389 

0.363 

0.336 

0.314 

0.400 

0.621 

0.560 

0.527 

0.491 

0.453 

0.424 

0.500 

0.782 

0.709 

0.669 

0.623 

0.575 

0.535 

0.600 

0.947 

0.863 

0.816 

0.762 

0.702 

0.650 

0.650 

1.030 

0.941 

0.891 

0.834 

0.768 

0.709 

0.700 

1.114 

1.021 

0.  968 

0.907 

0.837 

0.770 

0.750 

1.198 

1.102 

1.047 

0.983 

0.907 

0.834 

0.800 

1.  283 

1.183 

1.127 

1.060 

0.981 

0.901 

0.850 

1.369 

1.267 

1.209 

1.140 

1.058 

0.972 

0.900 

1.455 

1.  351 

1.292 

1.223 

1.139 

1.049 

0.950 

1.543 

1.436 

1.377 

1.  308 

1.  224 

1. 133 

0.975 

1.586 

1.479 

1.420 

1.351 

1.  268 

1.179 

I 
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TABLE  3 

OPTIMUM  OUTPUT  LO/UD  FACTOR  L^^j^  FROM  STATE  (i+1)  . 


bucket  size,  s 


L. 

1 

1 

2 

3 

5 

10 

25 

0.200 

0.024 

0.008 

0.003 

0.000 

0.000 

0.000 

0.300 

0.079 

0.034 

0.017 

0.005 

0.000 

0.000 

0.400 

0.178 

0.097 

0.059 

0.025 

0.004 

0.000 

0.500 

0.332 

0.208 

0.145 

0.080 

0.022 

0.001 

0.600 

0.548 

0.383 

0.295 

0.194 

0.085 

0.011 

0.650 

0.682 

0.497 

0.397 

0.281 

0.144 

0.030 

0.700 

0.833 

0.631 

0.521 

0.391 

0.231 

0.070 

0.750 

1.002 

0.784 

0.666 

0.526 

0.  348 

0.145 

0.800 

1. 190 

0.958 

0.834 

0.687 

0.499 

0.  266 

0.850 

1.398 

1.154 

1.025 

0.875 

0.686 

0.444 

0.900 

1.624 

1.371 

1.241 

1.091 

0.909 

0.682 

0.950 

1.871 

1.611 

1.480 

1.  335 

1.166 

0.976 

0.975 

2.001 

1.739 

1.608 

1.466 

1.307 

1. 139 

OUTPUT  LOAD  FACTOR  AT  OPTIMUM  L*  I 


Fig.  7 Optimum  output  load  factor 

a function  of  input  load  factor  . 
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be  Hashed 


100000 

21460 

4867 


78540 


16593 

4377 


Buckets 


Allocated 


8759 

1850 


The  data  for  the  last  two  columns  above  are  derived  from 
Tables  2 and  3.  For  example,  for  L q=  . 90  , and  s = 10  from 
Table  2 we  get  A^=  1.139,  i.e.,  we  allocate  100000/1.139 
slots  or  8750  buckets  to  File  #1. 

In  comparing  this  scheme  with  conventional  hash  coding 
methods  it  must  be  noted  that  99.51%  of  the  records  are 
available  within  one  access  cycle.  Using  the  tables  in 
Van  der  Pool  fll,  13]  we  find  that  the  average  number  of 
accesses  to  file  3 is  1.837  with  a standard  deviation  of 
0.069.  However,  since  file  3 accesses  represent  only  4.37% 
of  records,  the  average  number  of  access  cycles  over  all 
three  files  is  1.037  with  a standard  deviation  of  0.0145. 

In  Table  4 a comparison  is  provided  between  CHO,  open 
addressing  and  chaining  for  the  above  numerical  example. 

Further  improvement  in  performance  may  be  achieved  by 
either  increasing  the  number  of  files  or  decreasing  the  load 
factor. 
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TABLE  4 

COMPARISON  OF  TIMES  FOR  THREE  SCHEMES 


Concurrent 

Access 

Open 

Address  inqr 

Chaininq 

Mean  Access 

Cycles 

1.0366 

1.341 

1.210 

Standard 

Deviation 

.0145 

.011 

N/- 

# Concurrent 

Accesses 

3 

1 

1 

Load  factor 

.90 

.90 

.90  (in 
primary  only) 

.834 

(overall) 

Bucket  size 

10 

10 

10 

In  this  report  we  have  proposed  a file  access  algorithm 
(CHO)  based  on  concurrent  utilization  of  multiple  devices.  ^ 

We  have  developed  a mathematical  model  of  the  performance  of  j 

this  algorithm  and  have  demonstrated  how  this  model  may  be  I 

used  to  optimize  the  CHO  algorithm.  i 

I 

The  algorithm  developed  and  analyzed  here  is  only  one  ' 

I 

of  a class  of  related  algorithms  that  take  advantage  of 
existing  concurrency  in  computer/channel/l/o  device  systems. 

Many  airline  reservations  and  message  switching  systems 
handle  multiple  transactions  concurrently.  The  trandeoffs 
between  improved  throughput  over  many  transactions  at  the 
expense  of  single  transaction  response  time  in  those  systems 
and  opposite  characteristics  of  the  CHO  algorithm  described 
in  this  paper  need  to  be  examined.  Furthermore,  there  are 
variations  of  this  algorithm  not  yet  analyzed.  Nevertheless 
where  single  query  response  time  is  the  major  performance 
criterion  it  is  clear  that  significant  improvements  may  be 
effected  by  using  CHO  rather  than  searching  the  overflow 
records  sequentially. 
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Abstract 

In  this  report  we  describe  a new  file  organization,  cascade 
hashing,  which  uses  effectively  sequential  probes  to  search  for 
a record.  The  optimization  of  cascade  hashing  file  design  is 
formulated  as  a dynamic  programming  problem  which  is  solved  using 
a new  forward  procedure.  Some  results  on  the  comparison  of  cascade 
hashing  with  traditional  hash  file  organization  are  also  presented. 
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1.  INTRODUCTION 

A hash  file  Is  commonly  considered  the  beet  When  the  average 
number  of  probes  Is  mlnlmlzedt  Where  a probe  Is  the  operation  of 
reading  a bucket  and  examining  its  content  to  search  for  a par- 
ticular record.  This  criterion  is  adequate  for  files  stored  on 
truly  random  access  storage  media,  such  as  core  memory,  since  the 
probe  time  is  constant  on  such  media.  For  files  stored  on  disks, 
the  processing  time  is  a function  of  both  the  number  of  probes  cind 
the  time  for  each  probe.  Therefore,  even  When  the  average  number 
of  probes  is  minimized  to  be  close  to  one,  it  is  possible  to  reduce 
the  processing  time  further  by  reducing  the  probe  time. 

Basically,  there  are  two  kinds  of  probes  on  a disk  storage 
device,  random  probes  and  sequential  probes.  A random  probe  con- 
sists of  seeking  to  the  cylinder,  latency,  and  transmission.  A 
sequential  probe  consists  of  the  transmission  only.  Depending  on 
the  characteristics  of  disk  and  the  length  of  the  record,  a random 
probe  could  be  10  or  more  times  longer  than  a sequential  probe. 

In  this  situation,  the  number  of  probes  alone  is  not  adequate  for 
evaluating  the  performance  of  a hash  file. 

The  cascade-hash  file  organization  for  disk  files  proposed 
here  does  not  reduce  the  number  of  probes,  but  it  guarantees  that 
all  the  probes  after  the  first  one  are  effectively  sequential 
probes.  Therefore,  the  processing  time  is  reduced.  More  impor- 
tantly, for  the  same  average  processing  time,  a higher  load  factor 
may  be  used  so  that  less  storage  overhead  is  required  by  the  hash 
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file.  With  a slight  variation,  it  is  possible  to  do  the  probes 


in  parallel  to  reduce  the  processing  time  further.  Another 
tage  is  that  partial  reorganization  is  possible. 


advan- 


12-2 


I 


2.  THE  CASCADE  HASHING  METHOD 

2.1  The  cascade-hashing  concept 

In  a hash  file,  if  a separate  overflow  area  is  used,  any 
file  organization  may  be  used  to  store  the  overflow  records,  either 
treating  all  the  overflow  records  as  a whole  or  treating  them  in 
groups.  Let  A[l]  denote  the  primary  area  and  A [2]  the  separate 
overflow  area.  It  is  possible  to  store  the  overflow  records  from 
Atl]  as  a hash  file,  using  A [2]  as  the  primary  area  and  A[3]  , a 
third  area,  as  the  overflow  area.  Again  it  is  possible  to  store 
the  overflow  records  from  A [2]  as  a hash  file  using  A [3]  as  the 
primary  area  and  another  area  A [4]  as  the  overflow  area.  This 
"cascade  hashing"  may  continue  for  n stages  so  that  area  A[n]  is 
the  last  overflow  area. 

Although  any  file  organization  may  be  used  in  A[n]  , it  is 
perhaps  best  to  keep  it  a sequential  file  for  simplicity,  since 
only  a very  small  portion  of  the  records  are  expected  to  be  stored 
in  this  area. 


2.2  Cascade  hashing  for  disk  files 

To  adapt  the  above  cascade  hashing  method  for  disk  files, 
we  would  like  to  search  in  A [2]  , A [3]  , ...,  A[n]  serially  in 
such  a way  that  there  is  no  seeking  (read/write  assembly  movement) 
nor  any  latency.  The  search  in  A[l]  may  still  require  seek  and/or 
latency. 

Basically,  we  arrange  a chain  of  buckets  for  each  key  value 
so  that  the  buckets  are  (1)  in  different  areas,  (2)  on  different 


I 
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tracks  of  the  same  cylinder  and  (3)  at  consecutive  angular  posi- 
tions. In  most  cases,  the  switch  between  tracks  is  so  fast  that 
it  is  completed  before  the  next  bucket  is  read.  Therefore,  the 
probes  to  these  buckets  are  effectively  sequential  probes. 

Figure  1,  explained  below,  shows  how  this  is  realized. 

A number  of  cylinders  are  allocated  to  the  cascade-hash 
file.  For  head-per-track  disks,  we  consider  that  there  is  only 
one  cylinder.  A number  of  tracks  on  each  cylinder  are  allocated 
to  each  area  Afl],  At2],...,  and  Afn]  . A record  with  a key  value 
is  hashed  to  a bucket  in  A[l]  of  a certain  cylinder  by  a hashing 
function.  Let  the  address  be  (Cl,Tl,Bl),  where  Cl  is  the  cylinder 
address,  Tl  the  track  address  and  Bl  the  angular  position, 
expressed  as  a bucket  number  on  a track.  Note  that  the  value  Tl 
should  make  the  bucket  fall  in  area  A[l]  . 

The  next  bucket  for  overflow  records  from  the  home  bucket 
is  at  C2  = cl,  T2  = as  determined  by  a second  hashing  function 
(must  be  in  area  A [2])  , and  B2  = Bl+l  . The  bucket  for  overflow 
records  from  bucket  (C2,  T2,  B2)  is  C3  = C2,  T3  = as  determined  by 
a third  hashing  function  (must  be  in  area  A [3])  , and  B3  = b2+1  . 
Each  of  the  subsequent  buckets  is  determined  in  the  same  way  of 
(1)  keeping  the  same  cylinder,  (2)  hashing  to  a track  in  the  next 
area  and  (3)  increasing  the  angular  position  by  one  bucket. 

There  may  be  a "global  overflow  area"  A[n+1]  in  a separate 
area  to  store  the  records  that  may  overflow  from  A [n]  on  each 
cylinder. 
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Shovm  is  a 4-stage  process  in  a cylinder  with  4 buckets 
per  track.  Gaps  are  exaggerated.  Hashing  to  this 
cylinder  is  not  shown.  Another  cylinder  is  also  shown 
in  the  background. 


Figure  1 Cascade  Hashing 
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2.3  Operations  on  a cascade-hash  file 
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As  in  the  case  of  any  hash  file,  the  entire  area  allocated 
to  the  cascade-hash  file  must  be  initialized  with  deleted  records 
before  any  operation  may  be  performed. 

TO  add  a record,  an  empty  slot  (deleted  record)  is  searched 
for  in  the  series  of  buckets  described  before.  The  record  is  stored 
there  as  soon  as  an  empty  slot  is  found.  It  can  always  be  stored 
in  the  global  overflow  area,  which  degrades  the  performance  of  the 
cascade-hash  file. 

To  read  a record  with  a given  key  value,  it  is  searched  in 
the  series  of  buckets  described  above.  If  the  record  does  not 
exist  in  the  file,  every  bucket  in  the  series  plus  all  the  records 
in  the  global  overflow  area  must  be  searched.  This  indicates  that 
the  size  of  the  global  overflow  area  cannot  be  allowed  to  grow 
indefinitely.  Being  a hash  file,  the  cascade-hash  file  must  be 
re-organized  occasionally  if  the  size  of  the  (logical)  file  varies 
with  time.  However,  the  impact  of  increasing  number  of  records  is 
smaller  on  cascade-hash  files  because  more  stages  may  be  added 
without  seriously  degrading  the  performance. 

TO  delete  or  to  update  a record,  the  record  is  first  read 
then  changed  to  a deleted  record  or  to  the  record  reflecting  the 
desired  change.  If  the  key  value  of  a record  is  changed,  then  it 
must  first  be  deleted  and  then  added. 

I 

) 
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2.4  Considerations  on  Implementation 

The  global  overflow  area  A[n+1]  may  be  stored  using  any 
file  organization.  In  particular,  records  with  the  same  home 
address  may  be  chained  together  to  limit  the  search  for  a non- 
existent record.  If  more  tracks  in  each  cylinder  may  be  allocated, 
it  is  not  difficult  to  add  one  more  stage  to  the  original  cascade- 
hash  file.  Thus  unlike  the  traditional  hash  file,  partial  reorgan- 
ization is  possible. 

In  order  to  limit  the  search  for  a non-existent  record,  one 
extra  bit  may  be  allocated  for  every  bucket  (or  every  record)  to 
indicate  the  last  bucket  (or  record)  that  has  ever  been  stored  any 
valid  record.  This  bit  is  reset  only  by  an  add  operation  and  is 
set  only  at  initialization  or  reorganization  time  for  every  empty 
bucket  (or  slot).  Note  that  this  bit  must  not  be  changed  by  the 
delete  or  update  operation  after  initialization. 

If  the  areas  A[l],  Pi[2]  , . . . , Afn]  are  on  separate  packs, 
then  parallel  searching  is  possible,  since  all  the  addresses  of 
the  buckets  may  be  pre-computed.  This  is  a variation  to  the  gen- 
eral cascade-hash  file  organization. 

It  is  possible  to  use  hashing  with  open  addressing  [PETERSON 
1957]  in  the  local  overflow  area  A[n]  in  each  cylinder  instead 
of  using  the  sequential  file  organization.  This  is  not  advisable, 
since  it  complicates  the  searching  in  this  small  area  and  it  causes 
uneven  use  of  the  buckets  in  this  area.  The  latter  increases  the 
possibility  of  overflowing  from  a local  overflow  area. 
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The  records  with  the  same  home  address  (Cl#  Tl,  Bl)  may  be 
chained  together  to  limit  the  search  for  a non-existent  record. 
The  read  time  for  an  existent  record  is  not  affected  because  of 
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latency.  Note  that  chaining  must  be  done  at  the  record  level, 
since  each  bucket  must  store  records  with  different  home  addresses. 

It  is  sometimes  possible  and  more  advcintageous  to  let  hard- 
ware, e.g.,  l/O  controller  or  channel,  do  the  searching. 

The  inter-record  gaps  on  the  track  must  be  large  enough  so 
that  examination  of  record  key  values  in  the  current  bucket,  pos- 
sible computation  of  the  track  address  of  next  bucket,  and  switch 
to  the  new  track  can  be  finished  in  time  before  the  next  bucket 
arrives  at  the  read/write  head.  If  searching  is  controlled  by 
software,  there  may  be  problems  in  maintaining  control  over  the 
necessary  l/O  hardware. 

Since  n hashing  functions  with  different  ranges  are  used, 
it  is  advantageous  to  use  a single  algorithm  that  will  give  a 
hash  address,  using  the  range  as  an  additional  parameter  to  the 
key  value. 


-■  I 

■I 
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3.  OPTIMIZATION  OF  CASCADE-HASH  FILES  j 

I 3.1  Problem  definition  I 

I 

The  objective  of  this  section  is  to  formulate  a problem  for 

i 

finding  an  optimal  allocation  of  tracks  of  each  cylinder  to  storage 
areas  A[l]f  A[2],  A[n]  for  cascade  hashing  on  disk  storage 

devices.  A design  is  considered  optimal  if  it  minimizes  the  aver- 
' age  read  time  for  an  existent  record.  This  criterion  is  equivalent 

I to  minimizing  the  average  nutiiber  of  probes,  as  all  the  probes  I 

I except  the  first  one  are  sequential  probes,  whose  probe  times  do 

not  vary. 

^ Let  there  be  records  to  be  stored  in  an  area  of 

cylinders  on  a disk  storage  device.  Each  cylinder  has  tracks, 

buckets  or  = 

The  (over-all)  load  factor  p is  defined  as  , 

where  the  denominator,  denoted  by  S , is  the  total  number  of 
records  that  can  be  stored  in  the  whole  cascade-hash  file  area. 

The  above  variables  are  all  given  at  the  time  of  design  and  cannot 
be  varied. 

Design  Variable  - bucket  size:  A bucket  is  a unit  of  storage 
that  can  be  addressed,  i.e.,  a bucket  is  read  or  written  as  one 


Mj^_*b  records. 


each  capable  of  storing  Mg,p 


unit.  A bucket  may  be  chosen  to  contain  one  or  more  records.  The 
bucket  size,  b , is  the  number  of  records  that  a bucket  can  contain. 

Design  Variable  - number  of  stages:  There  are  n stages, 
from  A[l]  to  A[n]  . The  global  overflow  area  should  not  contain 
any  significant  number  of  records  for  a well-designed  cascade-hash 


file,  and,  therefore,  will  be  Ignored  for  considerations  of  optimal 
designs. 

Design  Variables  - area  sizes;  Within  each  cylinder,  a 
fixed  number  of  tracks  are  allocated  to  each  area.  Let  m[l] 
tracks  be  allocated  to  A[l],  m[2]  to  A [2],  ...,  and  m[n]  to 
A[n]  . Note  that 

n 

I ni[i]  = . 

i=l 

The  read  time  is  the  sum  of  the  first  random  probe  time  and 
the  probe  times  of  the  subsequent  sequential  probes.  For  cascade- 
hash  files,  minimizing  the  read  time  is  equivalent  to  minimizing 
the  number  of  probes. 

The  average  number  of  probes  is  computed  by  (1)  multiplying 
the  number  of  probes  for  each  bucket  by  the  number  of  records 
stored  in  it,  (2)  summing  up  these  products  for  all  buckets,  and 
(3)  dividing  the  sum  by  the  total  number  of  records. 

The  optimization  problem  then  is; 

given  p , , M 

adjust  b , n , m[l]  , m[2]  , . . . , m[n] 

to  minimize  average  number  of  probes. 

3.2  Notations 

The  following  is  a complete  list  of  all  the  symbols  used 
in  this  chapter. 

a : average  access  time  (unit  = time  to  transmit  a bucket)  of 

records  not  in  local  overflow  area. 
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A[i]  ; the  area  at  stage  1, 
b : bucket  size  in  records. 


n. 


C[i]:  l = number  of  tracks  available  for  allocation  to 

areas  A[l+1]  to  A[n]  . C[0]  ^ . c[n]  should  be  0 for 

an  optimal  design, 
b^  ; bucket  size  in  tracks. 

M : total  number  of  tracks  allocated  to  the  cascade-hash  file, 
number  of  buckets  per  track, 
total  number  of  cylinders  allocated. 

l=l,2,...«n.  number  of  tracks  allocated  to  area  A[i]  in 
a cylinder. 

number  of  records  per  track. 

number  of  tracks  allocated  to  the  cascade-hash  file  on  each 
cylinder. 

average  number  of  records  stored  on  each  cylinder, 
number  of  stages,  excluding  the  global  overflow  area. 
i = 0,l,  ...,n.  number  of  records  to  be  hashed  into  area 
A[i+1]  or  overflowing  from  area  Ati]  . N[0]  =N. 

total  number  of  records  in  the  cascade-hash  file, 
the  percentage  of  records  stored  in  Atn]  , the  local  over- 
flow area. 

p ; overall  load  factor,  the  ratio  of  total  number  of  records 
to  capacity  of  allocated  area  in  records.  p = n/S  . 

S ; capacity  of  allocated  area  in  records.  S = • 

t[i];  i=l,2,...,n.  read  time  contributed  by  records  stored  in 
area  A[i]  . 
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t*  ; minimum  average  access  time  for  given  n and  b (unit  = 

s 

revolution  time).  Its  value  is  computed  from  the  formula 
t*  = aL+ PovL(n- 1+0.5  (1/l+ PovN)]  , where  a and  Pov  maybe 
looked  up  in  the  tables  given  in  section  5. 
v(n,m,b);  number  of  overflow  records  when  n reads  are  hashed 

into  primary  area  of  m buckets,  each  capable  of  storing  b 
records. 

3.3  Further  conditions  and  observations  on  the  design  problem 

The  global  overflow  area  is  not  included  in  the  optimization 
problem,  since,  as  we  will  see,  the  number  of  records  in  each  local 
overflow  area  can  be  designed  to  be  almost  zero.  In  an  implementa- 
tion, this  area  must  be  provided  in  case  some  records  are  accidentally 
overflowed  from  a local  overflow  area. 

The  same  bucket  size  is  used  in  all  the  areas.  This  is 
required  of  areas  A[l],  A[2],  ...,  A[n-1]  to  achieve  sequential 
probing.  A smaller  bucket  size  in  the  local  overflow  area  A[n] 
may  improve  the  processing  time  to  some  extent-  However,  its 
effect  should  be  minimal,  since  only  a small  number  of  records 
are  stored  in  a local  overflow  area. 

Any  integer  value  less  than  is  possible  for  bucket 

size  b and  the  number  of  buckets  per  track,  , is  given  by 

the  smallest  integer  that  is  larger  than  or  equal  to  M^^/b  . On 
certain  devices,  only  a portion  of  such  values  may  be  chosen.  In 
case  the  optimal  design  requires  an  impossible  bucket  size,  it  is 
straightforward  to  find  a next  best  value  with  a feasible  bucket 
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- Theoretical  Hashing  Functions 


consider  the  key  of  a record  to  be  processed  as  a random 
variable,  denoted  by  K . A “theoretical  hashing  function"  is 
* defined  as  a transformation  T on  the  random  variable  k such 

that  T(K)  is  a discrete  uniform  random  variable  over  the  primary 
area,  l.e.,  T(K)  assumes  each  bucket  address  In  the  primary  area 
with  equal  probability.  The  probeibility  measure  associated  with 
K may  represent  probability  that  each  key  value  appears  in  the 
file.  In  this  case,  the  theoretical  hashing  function  tends  to 
distribute  the  records  evenly  among  the  buckets  in  the  primary 
area,  for  most  files  that  may  happen. 

However,  for  any  given  file,  i.e.,  any  given  set  of  key 
values,  a definite  number  of  key  values  are  hashed  to  each  bucket. 
There  is  no  randomness  or  probability  involved.  The  assumption 
of  such  a theoretical  hashing  function  is  merely  for  providing  a 
reference  point  among  all  possible  hashing  functions  and  for 
simplicity  in  analysis.  Such  a hashing  function  is  assumed  in 
most  studies  on  hash  files  and  hashing  functions,  e.g.,  [VANDERPCXJL 
1972] . 

It  has  been  shown  [LUM  1970]  that  a theoretical  hashing  func- 
tion can  be  outperformed  by  a well-chosen  hashing  function  for 
real  files.  Thus  the  performance  of  a hash  file  using  a theoretical 
hashing  function  may  be  considered  the  worst  case  among  all  hash 

j files  with  decent  hashing  functions. 

j 

I 

1 
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The  Use  of  Means 


If  we  consider  an  address  generated  by  the  hashing  function 
as  random  variable,  the  number  of  records  hashed  to  each  cylinder 
is  also  a random  variable.  However,  for  any  given  file,  the  num- 
ber of  records  hashed  to  a cylinder  is  a constant-  A similar 
situation  exists  with  the  number  of  records  overflowing  from  each 
stage  of  the  cascade-hashing  process.  If  we  treat  the  numbers  of 
overflow  records  as  random  variables,  the  optimal  design  obtained 
will  be  optimal  among  all  possible  files  weighted  by  their  proba- 
bilities of  occurrence.  Note  that  for  any  given  file,  it  is  often 
possible  to  obtain  a design  that  performs  better  than  such  an 
optimal  design. 

In  the  following  analysis  we  will  assume  a theoretical  hashing 
function  but  we  will  use  means  for  numbers  of  overflow  records 
from  the  stages.  The  optimal  designs  so  obtained  may  be  considered 
as  designs  that  one  should  usually  choose  without  being  given  a 
specific  file,  and  these  designs  are  "typical"  optimal  designs. 

We  can  expect  to  find  a better  design  if  the  characteristics  of 
the  given  file  do  not  change  much  during  its  life  and  such  char- 
acteristics can  be  taken  advantage  of. 

The  mean  number  of  overflow  records,  v , depends  on  the 

number  of  data  records  in  the  file,  n , the  number  of  buckets,  m , 

and  the  bucket  size,  b . Since  theoretical  hashing  function  is 

used,  the  number  of  records  N hashed  to  a bucket  follows  binomial 

B 

distribution  with  parameters  n and  1/m,  i.e., 

I 

j 


Prob(Ng  = i)  = cjp^(l-p)""^ 


where  p = l/m  . 

The  number  of  overflow  records  from  each  bucket  is  given  by. 


N =0  if  N„<b  and  N = N„  - b if  >b  ; the  mean  number  of 
V B V B B 


overflow  records  from  each  bucket  is  thus  given  by 


(l/m)*v(n,m,b)  = ^ (i-b)c"  p^  (1-p)""^ 

i=b+l 


V /•  i/1  x^-i  ^ ....^n  i,,  xO-i 

= 2^  (i-b)Cj^p  (1-p)  - 2,  (i~t>)C^p  (1-p) 

i=0  i=0 

b 

= n*p-b+  ^ (b-i)C?  p^(l-p)’^“^  . 


i=0 


Therefore, 


b-1 


v(n,m,b)  = m(n/m-b+  ^ (h- j )C " p^  (1-p)  ) . 

j=0 


Because  the  means  are  used,  it  is  sufficient  to  consider 


only  one  cylinder,  with  N = S-  p/M^  records  to  be  stored  in  it. 


The  mean  number  of  overflow  records  is  the  same  for  every  bucket 
in  each  area. 


Further  Assumptions; 

It  is  assumed  that  each  record  in  the  file  is  accessed  with 
equal  probability  when  computing  the  mean  read  time. 

The  unit  of  time  will  be  the  revolution  time  of  the  disk 
storage  device.  The  read  time  will  not  include  the  seek  time  and 
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latency  time  before  reading  the  first  bucket,  since  these  times 
are  the  same  for  all  cascade-hash  files  eind  hence  is  irrelevant 
for  finding  the  optimal  designs. 

3.4  The  optimization  problem 

The  central  problem  of  finding  optimal  designs  is  the 
optimization  problem  depicted  in  Figure  2.  in  the  figure,  the 
blocks  represent  transof rmations  at  stages  and  the  arrows  repre- 
sent the  input/output  status  of  variables.  Each  output  variable 
is  a function  of  all  the  input  variables  of  a block.  The  relation- 
ships among  these  variables  are  derives  as  shown  in  Figure  2. 

By  the  assumptions  made  earlier,  N records  are  to  be  stored 
in  tracks.  Let  us  consider  one  stage  (i.e.,  area)  at  a time 

to  estimate  the  total  read  time,  which  is  N times  the  mean  read 
time.  The  total  read  time  is  obtained  by  adding  up  the  read 
times  for  the  records  stored  in  all  the  stages. 

At  stage  1,  N[0]  =N  records  are  to  be  hashed  and  C[0]  = 

tracks  are  available.  If  we  allocate  m[l]  tracks  to  area  A[l]  , 
then  C[l]  =C[0]-m[l]  tracks  are  available  for  stages  2 and  on. 
Since  there  are  m[l]  • buckets  in  A[l]  , the  number  of  over- 

flow records  from  this  stage  is  N[l]  =v(N[0],  m[l]*Mg^,b)  , where 
V is  the  function  given  before.  The  total  read  time  contributed 
by  records  stored  in  A[l]  is  the  product  of  the  number  of  records, 
N[0]-N[1]  , and  the  read  time,  1/Mg^  revolutions.  Thus, 
t[l]  = (NtO]-N[l]  )/Mg^  . 
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At  stage  2,  there  are  N[l]  records  to  be  hashed  and  C[l] 
tracks  available.  If  we  allocate  m[2]  tracks  to  area  A [2]  , 
then  C[2]  =c[l]-in[2]  tracks  are  left  for  s\absequent  stages. 

Since  there  are  m[2]  • buckets  in  A[2]  , the  number  of  over- 

flow records  from  A [2]  is  N[2]  = v(N[l]  ,m[2] . The  total 
read  time  contributed  by  records  stored  in  A [2]  is  the  product 
of  the  number  of  records  N[l]-N[2]  and  the  read  time  2/Mg^ 
revolutions,  that  is,  t [2]  = 2 • (n  [1] -N  [2]  ) . 

In  general,  at  stage  i,  N[i-1]  records  are  to  be  hashed 
and  C[i-1]  tracks  are  available,  for  i=l,2,...,n.  Area  A[i] 
is  allocated  m[i]  tracks,  leaving  C[i]  =C[i-l]-m[i]  tracks  for 
subsequent  stages.  Note  that  l<m[i]<C[i]  for  i=l,  2 , . . . , (n-1) 
and  that  m[n]  =c[n]  . The  number  of  overflow  records  from  area 
Afi]  is  Nfi]  = v{N[i-l]  ,m[i] -Mg^,b)  , for  i=l,  2,  . . . , (n-1)  and 
N[n]  =0  . The  read  time  contributed  by  records  stored  in  area 
A[i]  is  given  by  t[i]  = n- (N [i] -N [i-1] ) revolutions,  for 
i=l,2,..., (n-1)  . The  read  time  for  a record  in  A[n]  is  more 
involved. 

The  read  time  for  a record  in  A[n]  is  the  sum  of  the  time 
to  search  (n-1)  buckets  sequentially,  the  rotational  delay  to 
reach  the  first  bucket  in  A[n]  and  the  time  to  search  in  the 
sequential  file  in  A[n]  . See  Figure  1.  This  is  given  approximately 
by  t[n]  = N[n-1]  ( (n-1)  + Mg,j,/2  + (1+N  [n-1]  ) / (2 -b)  ]/Mg^  . The  approx- 
imateness comes  from  the  fact  that  information  on  disk  storage 
device  is  read  in  units  of  buckets  but  this  equation  is  based  on 
the  assumption  that  reads  are  in  units  of  records.  Since  the 
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4.  THE  SOLUTION  STRATEGY 


4.1  The  dynamic  programming  problem 

For  a given  pair  of  (n,b)  , the  above  optimization  problem 

may  be  formulated  as  a dynamic  programming  problem.  Since  the 

numbers  of  feasible  values  in  n and  b are  small,  it  is  possible 

to  do  an  exhaustive  search  in  the  space  of  n and  b , solving  a 

dynamic  programming  problem  for  each  pair  of  values  of  n and  b . 

, , n and  b be  given.  We  will  formulate 

the  dynamic  programming  problem  and  describe  the  solution  strategy. 

n 

Let  f . (N [i-1] , C [i-1] ) be  the  minimum  of  E t[j]  by 

^ j=l 

adjusting  m[i] , m[i+l] , ...»  m[n]  . The  Principle  of  Optimality 

stages  f.  (N[i-1],  C[i-1])=  MIN  { t [ i]  + f . . (N  [i]  , C[i])}, 

^ l<m[i]  <C[i-l] 

i=l,2, ,(n-l),  where  N [0]  = N , C [0]  = , and  t[i]  is  a func- 

tion of  m[i]  and  N[i-1]  . For  i=n  , f^(N[n-l],  C[n-1])  =t[n] 
and  m[n]  = C [n]  . 

The  following  is  a summary  of  the  relations  derived  before. 


Let  N . 


t[i]  = n.  (N[i]-N(i-1]  )/Mg^  , for  i=l,  2,  . . . , (n-1)  , and 
t[n]  = ((n-1)  + 0.5Mg^+ 0.5(1+N(n-1]  )/b)-Ntn-l]  )/Mg^  . 

4.2  The  traditional  algorithm 

Traditionally,  a dynamic  programming  problem  is  solved 
"backwards"  by  computing  f^(Ntn-l])  for  all  possible  values  of 
N[n-1]  and  C[n-1]  , and  then  g^_j^  (N  [n-2]  , C[n-2])  for  all 
possible  values  of  N[n-2]  and  C[n-2]  , and  then  f^_2 (N [n-3] , C [n-3] ) 
for  all  possible  values  of  N[n-3]  and  C(n-3l  using  the  just  com- 
puted values  for  » and  so  on,  until  fj^(N[0])  is  obtained  by 

using  values  of  f2(N[l],  C[l])  . One  minimization  problem  has  to 
be  solved  for  computing  each  value  of  f^  . 

In  general,  not  all  the  f^  values  computed  will  be  used, 
making  some  of  the  computation  efforts  wasted.  In  the  problem  at 
hand,  the  values  of  N[i]  could  be  in  the  thousands  and  the  value 
of  Cfi]  could  be  in  the  hundreds,  thus  hundreds  of  thousands  of 
minimization  problems  have  to  be  solved,  although  most  of  the 
values  of  N[i]  and  C[i]  will  not  occur,  especially  at  later 
stages  (near  n) . In  order  to  avoid  unnecessary  computation,  an 
ingeneous  scheme  must  be  developed  to  estimate  the  possible  values 
of  N[il  and  C[i]  at  each  stage  and  solve  f^  for  these  values 
only.  If  later  a value  of  f^  is  found  to  be  needed  but  was  not 
computed,  one  must  be  able  to  re-compute  it,  possibly  triggering 
computation  of  or  even  f of  later  stages. 

However,  there  is  a new  forward  algorithm  which  will  auto- 
matically compute  only  the  needed  values  for  f^  , with  very  little 
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4.3  The  new  forward  algorithm 

Basically,  the  new  forward  algorithm  involves  applying  the 
Principle  of  Optimality  stated  above  for  i=l, 2, . . . , (n-1)  , in 
that  order. 

TO  compute  f (N  (0)  , C (0)  ) a search  method,  such  as  binary 
search,  is  used  to  find  the  optimal  value  for  m[l]  . For  a given 
m[l]  , the  values  of  t[l],  N[l],  and  C[l]  are  computed,  then  the 
computation  of  f^^  is  suspended  until  f 2 (N  [1]  , C [1]  ) is  computed. 
After  f2  (N[l]  ,C  [1]  ) is  computed,  the  computation  of  f (N(0)  ,C  (0) 
continues  until  the  optimal  value  of  m[l]  is  found  by  the  search 
method. 

To  compute  a value  of  f 2 (N [ 1] , C [ 1] ) , a search  method  is 
used  to  find  the  optimal  value  for  m[2]  . Similar  to  the  previous 
procedure,  an  f ^ (N [2] , c [2] ) is  required  for  each  trial  value  of 
m[2] , and  the  computation  of  f2 (N [1] , C [ 1] ) and  the  search 
algorithm  is  suspended  until  the  particular  f^ (N [2] ,c [2] ) is 
computed. 

The  computation  of  f^ (N [n-lj , C [n-1] ) is  straightforward. 

To  suspend  a computation  means  storing  all  the  variables 
and  the  point  of  suspension  so  that  the  computation  may  resume 
later  as  if  the  suspension  did  not  occur. 

In  the  above  procedure,  it  may  be  advantageous  to  store  the 
computed  values  of  f ^ (N [i-1] , C [ i-1] ) in  order  to  avoid  duplicate 
computation  in  case  the  same  value  of  f ^ (N [i-1] , C [i-1] ) is 
required  more  than  once. 
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The  computer  implementation  of  the  new  forward  algorithm  is 
straightforward  in  languages  such  as  AL30L  and  Pl/I  which  provide 
recursive  procedures.  Both  the  dynamic  programming  procedure  cuid 
the  minimization  procedure  must  be  recursive  (i.e.,  being  able  to 
call  itself) . 

4.4  comparison  of  the  two  procedures 

Since  the  new  forward  algorithm  automatically  avoids  unneces- 
sary computation  of  unused  f^  values,  it  can  be  expected  that  the 
forward  algorithm  requires  less  computation.  But  then  the  storage 
requirement  could  be  more. 

As  an  example,  assume  there  be  50  records  to  be  stored  and 
there  be  20  tracks  on  one  cylinder.  If  the  backward  algorithm  is 
used,  1000  values  of  f^  need  to  be  computed  at  each  stage.  This 
means  4,000  values  in  total  for  a 5-stage  problem. 

On  the  other  hand,  if  the  new  forward  algorithm  is  used, 
the  number  of  f^  values  computed  is  much  less.  Assume  4 tracks 
are  allocated  at  each  stage  then  about  (log2  20)  (log2  16 ) , 

(log2  12)  • • • (log2  4) , or  about  380  f^  values  will  be  computed. 

If  the  computed  f^  values  are  stored,  then  the  number  will  be 
even  less. 

In  general,  small  number  of  stages,  large  N and  , 

expensive  minimization  and  relatively  inexpensive  storage  favor 
the  new  forward  algorithm.  The  programming  cost  for  the  two 
algorithms  is  about  the  same.  After  considering  these  factors, 
the  new  forward  algorithm  is  chosen  for  finding  optimal  cascade- 
hash  file  designs. 
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4.5  The  adopted  algorithm 

, t,  n,  N,  and  the  following 

algorithm  is  used  to  compute  the  minimum  access  time  and  the 
optimal  design,  namely,  optimal  values  ofr  m(l] , m[2] , ...,  m[n]  . 

We  assume  that  the  algorithm  is  implemented  as  a recursive 
procedure  names  DP(i,N,C)  returning  as  its  result  the  minimum 
access  time  for  storing  N records  in  C tracks  at  stage  i (with 
n-i  stages  remaining).  The  procedure  stores  its  findings  as  an 
entry  (i,N,C, m*, t*)  in  a 5-column  table,  called  f-table,  where  m* 
and  t*  are  the  optimal  values. 

We  also  assume  available  a minimization  procedure  MIN (D,OBJ, d* ) 
which  finds  a d*  such  that  l<d*^D  and  OBJ(d)  is  minimized, 
where  OBJ  is  a procedure  which  returns  a result.  The  procedure 
MIN  must  be  recursive.  In  our  design  problem,  OBJ (d)  gives  as 
its  result  t[i]  , the  total  access  time  of  the  records  stored  in 
stage  i if  d tracks  are  allocated  to  A[i]  , and  D is  C[i]  at 
stage  i . 

The  procedure  DP(i,N,C)  is  described  below. 

(1)  If  N >b*C*Mg^  , then  return  with  "infinity;"  otherwise, 
continue. 

(2)  Search  in  the  f-table  for  (i,N,C,-,-).  If  found,  then 
return  with  t*  ; otherwise,  continue. 

(3)  compute  t*  = MIN (C,OBJ, d)  , where  OBJ (m)  = n-  (N-N' ) /m^^  + 

DP  (i+l,N' ,C-m)  and  N'  = v (N,  m-Mg^,,b)  . The  golden-sectioning 
method  is  used  in  MIN- 


For  given  values  of  M^^^ 
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(4)  Set  m*  = d , where  d is  given  by  MIN. 

(5)  Add  an  entry  (i,N, C,  tn* , t* ) to  the  f-table. 

(6)  Return  with  t*  . 

TO  compute  the  minimum  access  time,  initialize  the  f-table, 
compute  *=  » where  Mg^  must  be  an  integer,  and  then  call 

DP(i,N.M,j,^)  . 

The  above  algorithm  has  been  implemented  in  PL/1.  These 
programs  are  used  to  compute  the  optimal  cascade-hash  designs. 


5.  OPTIMAL  DESIGNS 


The  optimal  designs  for  all  practical  values  of  n,  b,  and 
p have  been  obtained. 

The  average  access  time  is  composed  of  two  terms,  the  first 
term  reflecting  the  access  time  contributed  by  records  in  areas 
A[l],  A[2] , A[n-1] , and  the  second  term  reflecting  the  access 

time  contributed  by  records  in  A[n]  , the  local  overflow  area. 

The  first  term  is  a product  of  the  block  length,  L = b/Mj^^  , 
(block  size  in  tracks)  and  a value  which  is  a function  of  the 

, block  size  b , and  the  number  of 
stages,  n , only,  assuming  the  time  unit  to  be  the  revolution 
time — the  time  to  read  one  track  of  data. 

The  second  term  can  be  expressed  as  a function  of  n,  b^,  S 
(total  number  of  records  that  can  be  stored  in  all  areas),  and 
Pov  (the  percentage  of  records  in  the  local  overflow  area) , where 
Pov  is  a function  of  n and  b only. 

From  the  computed  results,  the  minimum  average  access  time 
obtained  by  solving  a dynamic  programming  problem  as  described 
before  may  be  expressed  as  t*  = a • b^  + Pov  • b^^  (n-1  + 0.5  (1/l+Povn)  ) , 
where  a and  Pov  are  functions  of  p,  b,  and  n only,  and  are 
given  in  Table  1. 

The  optimal  designs  for  various  values  of  R,  b,  and  n are 
given  in  Table  2,  assuming  = 50  tracks.  If  is  not  50,  the 

tracks  should  be  allocated  proportionally. 


load  factor,  p=n/(M,j,^ 
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Most  of  the  results  shown  in  the  tables  below  have  been 
obtained  by  assuming  S = 1000  records,  = 50  tracks,  and  - 20 

records  per  track,  and  some  of  them  have  been  checked  with  differ- 
ent values  of  N and  . 


i 

1 
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(2)  Load  factor  = 90% 


2.02  11.0  1.72 

1.80  3.0  1.62 

1.42  0.6  1.27 

1.36*  0.0  1.35  1.1  1.26 

1.18*  0.0  1.18*  0.0  1.18 

1.09*  0.0  1.09*  0.0  l.iol  0.3 


(3)  Load  factor  = 85% 


(4)  Load  factor  = 80% 


1.87  4-5  1.66 

1.51  0.6  1.47 

1.25*  0.0  1.26 
1.19*  0.0  1.19* 
1.08* 


10.1  1.35 

2.1  1.29 
0.3  1.22 
0.0  1.18 


21.5  1.00 

9.1  1.00 

2.3  1.00 

1.4  1.00 


0.0  1.08  0.4  1.00 

1.03*  0.0  l.OOl  3.0 


(5)  Load  factor 


1.62 
1.  39 

1.19*  0.0 
1.14*  0.0  1.16 
1.06* 
1.02* 


17.7  1.00  38.7 
6.7  1.00  24.9 
1.3  1.00  14.1 
0.7  1-00 
0.0  1.00 
0.0  1.00 


(6)  Load  factor 


6 

a 

POV 

a 

Pov 

1.74 
1.  33* 
1.14* 

0.6 

0.0 

0.0 

1.70 
1.33* 
1.14* 
1. 10* 

1.58  5-3  1.33  15.0  1.00  35.3 

1.32  0.9  1.24  4.7  1.00  21.6 

1.14*  0.0  1.15  0.7  1.00 

1.10*  0.0  1.12  0.3  1.00  8. 

1.03*  0.0  1.00  3. 

1.01*  0.0  1.00  0.9 


TABLE  1 


MINIMUM  AVERAGE  ACCESS  TIME 
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(1)  Load  factor  = 95% 

Total  number  of  tracks 


b = 1 


1 

5 4 3 2 1 

m[i] 

16  12  8 5 9 

16  12  9 13 

18  13  18 

21  29 

= 50 


b = 2 


m 

5 4 3 2 1 

m[i] 

21  12  8 5 4 

22  13  8 7 

24  14  12 

27  23 

t = 5 


B 

6 5 4 3 2 1 

m[i] 

30  13  5 1 1 0 

32  11  4 2 1 

28  14  6 2 

31  14  5 

35  15 

b = 10  b = 20 


B 

5 4 3 2 1 

m[i] 

43  6 1 0 

41  8 1 

44  6 

B 

5 4 3 2 1 

m[i] 

39  8 2 1 0 

36  10  3 1 

36  12  2 

40  10 

TABLE  2 OPTIMAL  ALLOCATION  OF  TRACKS 


12-30 


TABLE  2 Optimal  Allocation  of  Tracks  (continued) 
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TABLE  2 OPTIMAL  ALLOCATION  OF  TRACKS  (continued) 


(6)  Load  factor  = 70% 

Total  number  of  tracks  = 50 


TABLE  2 OPTIMAL  ALLOCATION  OF  TRACKS  (continued) 


6.  DISCUSSION 


1.  Some  general  observations 

The  access  time  is  the  sum  of  two  components,  one  contributed 
by  records  stored  in  areas  other  than  the  overflow  area  (regular 
component)  and  the  other  contributed  by  records  stored  in  the  over- 
flow area  (overflow  component) . 

The  regular  component  is  proportional  to  the  bucket  size  and 
a factor,  a , that  is  a function  of  the  design  variables.  The 
factor  a varies  much  less  them  the  overflow  component  does,  for 
optimal  designs. 

The  overflow  component  increases  linearly  with  the  total 
nuniber  of  records  and  increases  quadratically  with  the  percentage 
of  records  in  overflow  area.  Therefore,  given  a constant  number 
of  records  in  the  file,  if  the  number  of  records  in  the  overflow 
area  may  be  minimized  to  approach  as  closely  as  possible  to  zero, 
the  overflow  component  is  minimized.  Two  ways  of  doing  this  are 
to  increase  the  bucket  size  and  to  increase  the  number  of  stages. 
Neither  one  increases  the  regular  component  significantly. 

A fact  to  be  remembered  is  that,  at  stage  i , the  number  of 
records  that  overflow  to  next  stage  (i+1)  is  proportional  to  the 
number  of  records  loaded,  N(i-l)  , and  decreases  with  a larger 
bucket  size  and/or  a smaller  local  load  factor,  N [i-1] /m[i] . 

2.  The  effect  of  number  of  stages 

Since  a fraction  of  the  loaded  records,  N[i-1]  , is  stored 
at  every  stage  the  number  of  records  that  overflow  to  next  stage. 
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N[i]  / decreases  roughly  geometrically  with  i . Therefore,  the 
more  stages  there  are  the  less  records  are  stored  in  the  overflow 
area  and  the  smaller  the  overflow  component  is,  hence  the  smaller 
the  access  time  is.  But  when  the  number  of  overflow  records  has 
become  effectively  zero,  the  access  time  does  not  vary  with  the 
number  of  stages  any  more. 

The  results  presented  in  the  tables  in  previous  sections 
indeed  show  that  when  the  number  of  stages  increases  while  keeping 
the  bucket  size  constant,  the  optimal  access  time  decreases  \intil 
it  reaches  a minimum  and  stays  there.  This  minimum  is  closer  to 
the  absolute  lower  bound  for  larger  bucket  sizes.  This  lower 
bound  is  the  time  to  retrieve  one  bucket,  (b/M^^^)  revolutions. 

The  results  also  show  that  the  optimal  time  reaches  its 
minimum  faster  for  greater  bucket  sizes.  On  the  other  hand,  the 
optimal  time  reaches  the  minimum  slower  with  a greater  load  factor. 
This  is  explained  by  the  fact  that  a smaller  percentage  of  records 
overflow  to  next  stage  if  the  bucket  size  is  larger  eind/or  the 
local  load  factor  is  smaller.  For  optimal  designs,  the  local  load 
factors  are  monotonic  functions  of  the  overall  load  factor. 

3.  The  effect  of  bucket  size 

As  described  before,  increasing  the  bucket  size  decreases 
the  number  of  records  in  the  overflow  area,  making  the  access  time 
closer  to  the  lower  bound,  . This  generally  decreases  the 

access  time.  However,  increasing  the  bucket  size  increases  the 


I 
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lower  bound  also.  This  is  reflected  in  the  data  by  the  access 
time's  first  increasing  then  decreasing  with  the  increasing  bucket 
size,  while  keeping  all  other  variables  constant  and  the  alloca- 
tion optimal. 

If  the  difference  between  the  optimal  access  time  and  the 
absolute  lower  bound  is  plotted  against  the  bucket  size,  a strictly 
decreasing  curve  results. 

When  there  is  no  limitation  on  the  number  of  stages,  the 
optimal  access  time  is  very  close  to  the  lower  bound,  • which 

is  minimized  by  the  smallest  possible  bucket  size. 

4.  Conclusion 

When  the  searching  is  not  interrupted  by  other  activities 
on  the  system  and  there  is  no  limitation  on  the  number  of  stages, 
the  bucket  should  be  as  small  as  possible.  This  is  the  case  vhen 
the  searching  in  the  cascade-hash  file  may  be  done  by  one  initiate 
of  l/O  command. 

If  the  number  of  records  in  the  file  exceeds  what  the  file 
organization  is  designed  for  and  it  is  too  expensive  to  reorganize 
the  whole  file,  it  is  possible  to  reorganize  only  the  overflow  file 
into  an  additional  stage  or  to  reorganize  only  the  last  few  stages, 
which  are  usually  small. 
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Abstract 


APL-STARAN  Is  a dialect  of  APL  designed  to  be  more  amenable  to  compilation  | 

than  the  standard  version  of  APL.  The  principle  differences  include  the  use  ? 


of  name  prefixes,  the  ability  to  accept  a limited  character  set  for  denoting  ! 

I 

the  primitive  functions,  some  variations  and  restrictions  on  the  use  of  the  i 

program-branching  primitive,  and  the  employment  of  a subset  of  APL  functions 
with  some  additional  I/O  primitives.  The  reasons  for  each  of  these  modifications 
are  discussed  in  detail,  as  well  as  the  implications  for  transportability 
between  the  two  dialects.  An  implementation  of  APL-STARAN  has  been  completed 
for  the  STARAN  Associative  Processor.  An  outline  of  this  implementation  as 
well  as  a user's  guide  and  report  of  actual  system  performance  are  presented. 
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SECTION  1:  THE  DESIGN  OF  APL-STARAN 


APL  as  currently  defined  by  the  APL/360  implementation  (see  Refs.  [1] 
and  [2])  seems  to  be  a natural  candidate  for  a high-level  language  for  SIMD- 
type  parallel  processors  [3]  - [5],  For  a number  of  reasons,  however,  it 
does  not  appear  that  it  can  be  conveniently  compiled.  For  instance,  the 
context-sensitive  nature  of  a number  of  primitive  symbols  leaves  their 
semantics  ambiguous  until  execution  time.  Specifically,  there  are  a group 
of  primitive  symbols  v;hlch  are  used  to  denote  two  completely  different 
functions  depending  on  the  number  of  arguments  supplied  with  the  symbol. 

An  example  would  be  the  cross  (ordinary  multiplication  sign),  which 
can  be  used  monadically  (i.e.,  with  only  a right  argument)  to  represent  the 
signum  function,  or  it  can  be  used  dyadically  (i.e.,  with  both  a left  and 
right  argument)  to  denote  the  multiplication  function.  In  any  given 
context  the  precise  function  intended  can  be  determined  by  examining  the 
source  code  to  the  left  of  the  s5rmbol.  If  there  is  some  primitive  scalar 
function  on  the  left,  then  the  cross  has  no  left  argument  and  hence  is  the 
signum  function.  If  there  is  a right  parenthesis  or  a number  to  the  left 
of  the  cross,  then  it  is  known  to  be  dyadic  and  thus  is  the  multiplication 
function.  For  these  cases  the  meaning  is  uniquely  determined  when  the 
source  code  is  entered.  Such  an  arrangement  allows  compilation,  provided 
one  omJ.ts  consideration  of  the  memory  management  required  for  storage  of 
the  result. 

The  complication  sets  in  when  a name  or  identifier  appears  to  the  left 
of  the  cross.  There  is  no  way  of  knov;ing,  prior  to  the  actual  execution  of 
the  statement,  whether  this  name  will  represent  (at  execution  time)  a 
variable  value  or  a monadic  defined  function  call  - and  that  makes  all  the 
difference.  Another  difficulty  in  compiling  ATL  stems  from  the  absence 
of  declaration  statements  in  the  language.  This  makves  memory'  management  a 
verj’  complicated  process. 

An  obvious  response  to  these  difficulties  is  to  run  AFL  interpretative!/. 
Not  only  does  this  solve  the  problems  inentioneJ,  but  it  also  buys  a number 
of  little  extras  that  are  very  convenient  for  interactive  programming.  On 
the  other  hand,  it  entails  a certain  loss  of  e;:(  cution  efficiency.  Tliis  is 
counter-productive  to  the  increased  computational  throughput  expected  from 
parallel  processing. 
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The  solution  that  Is  proposed  here  is  to  Introduce  a dialect  of  APL  (to 
be  called  APL-STARAN)  which  can  be  more  readily  compiled^  yet  will  remain 
as  close  as  possible  to  the  basic  structure  of  standard  AFL.  This  permits 
parallel  processing  programs  to  be  Interactively  written,  tested  and  debugged 
on  existing  APL  systems,  and  then  transferred  to  the  parallel  processing 
systems  for  production  use.  It  also  provides  the  parallel  processing 
programmers  with  a tested  set  of  array  oriented  algorithms  that  have  been 
developed  by  the  AFL  ptogrammers. 

PeslRn  Goals  and  Tlielr  Rationale 

There  appear  to  be  two  solutions  to  the  problem  of  using  the  same 
symbol  to  represent  two  different  functions,  namely  (1)  change  one  of  the 
symbols,  or  (2)  provide  naming  conventions  which  explicitly  identify  the 
class  of  a name  used  on  the  left  of  the  ambiguous  symbol.  The  first 
solution  is  the  obvious  one,  but  it  is  subject  to  the  serious  objection 
that  it  would  be  difficult  to  transport  programs  between  the  two  dialects 
of  APL.  For  this  reason  the  second  of  the  proposed  solutions  was  chosen 
for  APL-STARAN.  This  also  solves  the  related  problem  of  detecting  the  syn- 
tax error  in  an  attempted  assignment  of  a value  to  a nlladic  function  name 
or  a label. 

All  names  in  APL-STARAN  are  given  alphabetic  prefixes  which  specify  the 
name-class  of  the  object  identified.  There  are  five  name-classes,  and  each 
is  distinguished  by  a two  letter  prefix  as  shown  in  Table  I.  This  means 
that  all  APL-STARAN  names  are  a minimum  of  three  characters  long.  The 
first  two  must  be  a proper  prefix  and  the  remaining  characters  may  be  any 
of  the  alphanumerics.  Such  a procedure  causes  these  names  to  form  a sub- 
set of  APL  names.  Provided  the  names  used  in  standard  APL  programs  are 
chosen  from  that  subset,  all  programs  can  be  interchanged  without  alteration 
between  the  two  dialects. 

In  order  to  free  APL-STARAN  from  the  need  of  having  the  special  I/O 
devices  necessary  to  generate  the  standard  APL  graphics,  the  APL-STARAN 
compiler  has  been  designed  to  accept  mnemonics  in  place  of  the  "non- 
ASCII-llkc"  APL  symbols.  There  have  already  been  a number  of  proposals  for 
such  a set  of  mnemonics  (see  Refs.  [6]  - [8]),  but  they  generally  employ  a 
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TABLE  I.  NAME  PREFIXES 


Prefix 


VA 

LA 

NF 

MF 

DF 


Name  Class 


Variable 

Label 

Nlladlc  function 
Monadic  function 
Dyadic  function 
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1 

I 

i 
1 

precediar,  escape  cluiracter  to  differentiate  the  mnemonics  from  a similarly 

constructed  APL  name.  In  addition,  the  escape  sequence  is  designed  to  be  > 

processed  by  the  system  input  routine.  This  means  that  the  conversion  from 

the  supplied  mnemonic  to  the  proper  internal  representation  is  made 

wherever  the  escape  convention  is  encountered.  In  other  words,  if  the 

escape  sequence  were  used  within  a character  literal,  it  would  be  converted 

by  the  system  to  the  appropriate  internal  representation  of  the  graphic 

for  which  it  was  the  mnemonic.  This  is  a suitable  arrangement  provided 

one  is  willing  to  allow  the  internal  character  count  of  some  arrays  to  be 

different  from  that  of  their  graphic  representations.  The  conventions 

adopted  for  APL-STAlli\N  have  avoided  this  anomaly. 

These  conventions  replace  the  "non-ASCII-llke"  APL  graphics  with  a two  , 

letter  mnemonic,  which,  because  of  the  prefixing  scheme  chosen,  cannot  be 
confused  with  a name.  These  mnemonics  are  entered  into  the  system  pre- 
cisely as  the  symbols  for  which  they  are  the  graphics  (not  as  the  symbols  , 1 

for  which  they  are  the  mnemonics).  The  difference  comes  when  the  compiler 
encounters  them.  At  this  point  they  arc  translated  into  the  proper  execution 
code  according  to  their  mnemonic  meaning.  The  mnemonics  recognized  bj'  the  . 

compiler  are  shown  in  Table  II.  In  cases  where  two  mnemonics  are  shoim 
for  a given  primitive,  both  have  been  provided  merely  to  improve  the  i j 

readibility  of  the  mnemonic  source  code,  but  they  may  be  used  interchange- 
ably. The  intended  meaning  is  determined  by  the  compiler  following  the 
same  procedures  it  would  use  if  the  primitive  graphic  had  been  employed. 

Using  this  set  of  mnemonics,  APL-STARAN  programs  may  be  entered  from 
virtually  any  I/O  device. 

Three  new  primitive  functions  have  been  added  to  allow  the  f''ll  use 
of  all  available  I/O  devices.  There  is  one  input  function  (IN)  which  is 
monadic  and  takes  a two  component  vector  for  its  right  argument.  Tliis 
argument  specifies  a dataset  number  which  is  the  logical  identifier  of  a 
physical  device  and  a specific  file.  The  assignment  of  a logical  identifier 
to  a given  physical  file  is  made  by  a command  nvailabi e in  the  DOS-11 
operating  system.  Tlie  result  generated  by  the  input  function  is  the 
value  of  the  data  object  identified  by  its  right  argument. 

There  are  two  functions  provided  for  generalized  output;  OB  is  the 
liinemonic  for  bare  output  and  OT  denotes  tcmnlnated  output.  These  functions 
are  identical  except  for  a final  new-line  character  (carriage  return  plus 
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TABLE 

II.  APL-STARAN  SYMBOLS  AND  MNEMONICS  (VI. 1) 

Primitive  Scalar  Functions 

SYMBOL 

MNEMONIC 

DYADIC  FUNCTION 

MONADIC  FUNCTION 

♦ 

Plus  (Addition) 

Identity  (Numeric  value) 

- 

- 

Minus  (Subtraction) 

Arithmetic  negation 

X 

TU 

SG 

Times  (Multiplication) 

Slgnum  (Sign  of) 

♦ 

DV 

Divide  by 

r 

MX, 

CE 

Max Imum 

Celling 

L 

MN, 

FL 

Mini  mum 

Floor 

1 

RS, 

AB 

Res  1 due 

Absolute  value 

* 

* 

Power 

Exponent i al 

• 

LG 

Base  logarithm 

Natural  logarithm 

< 

< 

Less  than 

& 

LE 

Less  than  or  equal 

8 

Equal 

* 

NE 

Not  equal 

i. 

GE 

Greater  than  or  equal 

> 

> 

Greater  than 

A 

AN 

And 

V 

OR 

Or 

At 

NA 

Nand  (Not  and) 

¥ 

NR 

Nor  (Not  or) 

~ 

NT 

Not  (Logical  negation) 

Primitive  Mixed  Functions 

SYMBOL 

MNEMONIC 

DYADIC  FUNCTION 

MONADIC  FUNCTION 

P 

RH 

Restructure  (Reshape) 

Size  (Shape) 

i 

10 

Least  Index  of 

Index  generate 

• 

Last  axis  catenate 

Ravel 

.n 

Catenate  or  Laminate 

Special  Functions  and  Symbols 

SYMBOL 

MNEMONIC 

FUNCTION  OR  MEANING 

-4- 

IS 

Ass  1 gnment 

-► 

GO 

Go  to  (branch  or  jump) 

IF 

Conditional  Branch  (used 

1 with  GO) 

[;] 

C;3 

Index  selector 

() 

() 

Grouping  symbols 

t 

f 

Quote  (character  string 

del imi ter ) 

an 

Hl-mlnus  (negative  number  Indicator) 

/ 

/ 

Vector  reduction 

I 

IB 

I-Beam  (system  dependent 

functions) 

IN 

Monadic  input  (from  file 

system) 

0 

Nl 

Niladic  input  (quote-quad  input) 

1 

EX 

Execute  (evaluate) 

D 

EX 

Nl 

Evaluated  input  (quad  input  v/lthout  prompt) 

OT 

Terminated  output  (monadic  or  dyadic) 

OB 

Bare  output  (monadic  or 

dyad i c ) 

7 

DL 

Del  (function  delimiter) 

• 

f 

• 

/ 

Local  variable  separator 

: 

• 

• 

Label  separator 
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line  feed)  that  is  suppl^Led  by  the  system  at  the  end  of  each  OT  output 
request.  Both  functions  are  essentially  dyadic  (with  conventions  for 
eliding  the  left  argument).  'Hie  left  argument  specifies  a logical  dataset 
(similar  to  the  input  function)  and  the  right  argument  provides  the  data 
object  to  be  output.  Like  other  APL  primitives,  the  output  functions 
generate  a result,  which  in  this  case  is  the  value  of  the  right  argument. 

In  other  words.  Ignoring  the  physical  output  produced,  these  functions 
act  as  if  they  were  the  identity  function.  An  alternate  way  of  viewing 
these  two  functions  is  .ss  a spcciellzed  case  of  the  assignment  function 
where  the  left  argument  is  the  logical  name  of  a shared  variable  [9]. 

Tlie  problem  of  memory  management  in  the  absence  of  dimension  declara- 
tions is  solved  by  subterfuge.  Although  subsequent  versions  of  APL-STARAN 
may  wish  to  attack  this  problem  directly,  the  present  version  solves  it 
by  a get-space  routine  that  docs  the  job  at  execution  time.  This  means 
that  there  is  not  a unique  correspondence  between  the  variable  names  and  j 

their  physical  memory  addresses.  Data  values  are  located  through  a data  : 

descriptor  table.  Once  seme  experience  is  gained  with  this  implementation 
it  will  be  possible  to  determine  if  a better  solution  to  the  problem  is 
demanded.  j 

There  has  been  an  improvement  in  the  efficiency  of  the  branching  mechan-  .. 
ism  supplied  with  standard  APL.  Since  the  whole  question  of  a proper  set  of  « 

control  structures  for  APL  is  under  investigation  (see,  for  example.  Refs.  j 

[10]  and  [11])  the  arrangement  described  here  should  be  seen  as  only  an  i 

interim  solution.  Although  the  GOTO  is  not  eliminated  (indeed,  it  remains  ! 

i 

the  kernel  of  the  whole  control  system)  it  is  made  subject  to  a number  of  ; 

restrictions  designed  to  provide  more  efficient  execution.  As  a by-product  j 

these  restrictions  significantly  improve  one's  ability  to  trace  the  flow  of 
control  through  a program. 

There  are  only  three  types  of  syntax  allowed  when  using  the  branch  ' 

arrow.  It  may  be  used  | 

(1)  with  a single  label,  (including  the  null  label  LA), 

(2)  with  an  indexed  label  list,  or  ^ ; 

(3)  with  a single  label  followed  by  the  mnemonic  IF,  followed  by  any  i 

AFL-STARAN  expression  that  evaluates  to  a 1 or  0. 
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This  means  that  the  target (s)  of  any  branch  is  (are)  always  visible  at  the 
source  level.  Although  this  is  more  restrictive  than  the  ordinary  APL 
usage,  it  avoids  the  possibility  of  a variable  name  or  function  name  complete- 
ly obscuring  the  destination  of  the  branch. 

The  indexed  label  list  mentioned  in  the  second  type  of  branch  is  con- 
structed by  simply  Juxtaposing  a number  of  labels  as  if  they  were  the 
elements  of  a numeric  vector,  and  then  indexing  this  string.  This  is  the 
format  required  by  the  compiler,  but  it  is  not  permitted  in  standard  APL. 
Nonetheless,  its  semantic  equivalent  may  be  easily  obtained  in  APL  by 
catenating  the  labels,  enclosing  them  in  parentheses  and  indexing  the 
resulting  expression.  This  multiple  branch  provides  the  "case  selection" 
capability  used  by  structured  programming  languages  to  execute  one  of  a 
number  of  alternate  program  blocks. 

The  third  type  of  branch  uses  a new  primitive  (IF)  which  is  nothing 
more  than  the  compression  function  with  its  arguments  reversed,  together 
with  the  restriction  that  its  left  argument  must  be  a single  label.  Although 
this  function  is  not  available  in  standard  APL  as  a primitive,  it  is 
easily  introduced  as  a user  defined  function.  Indeed,  there  are  no  real 
transportability  problems  between  the  two  dialects  because  of  these  branch- 
ing restrictions.  All  that  is  required  is  that  these  restrictions  also  be 
followed  in  the  standard  APL  programs. 

Basic  Structure  of  the  Iiapleiner.tation 

An  implementation  of  APL-STARAN  has  been  comple!  iid  for  the  STASAl^ 
Associative  Processor  at  Rome  Air  Development  Center.  The  compiler  is  itself 
written  in  APL-STARAN.  It  coiumunicates  with  the  I/O  devices  through  a super- 
visor module  written  in  MACRO-11  asricinbly  code.  This  module  runs  on  a 
PDP-1]  which  acts  as  the  sequential  control  unit,  for  SI.’APjVN.  The  supervisor 
module  uses  the  basic  I/O  facilities  for  the  DOS-11  batch  operating  nysyera, 
and  presents  the  compiler  with  a source  program  in  a form  which  is  basically 
ASCII. 

The  compiler  translates  the  source  program  into  an  object  module  in  a 
form  called  QCODE.  If  the  source  program  is  a function  definition,  then 
the  object  module  is  merely  stored  in  Llie  control  memory.  Otherwise  the 
compiler  .acts  as  a lo.ad-and-go  system,  deleting  the  object  module  whej) 
execution  is  cojiiplete. 


i 


QCODE  is  the  AFL-STARAN  execution  code,  and  Is  essentially  a form  of 
threaded  code  112].  The  QCODE  machine  Is  a stack  oriented  virtual  machine 
composed  of  two  main  modules,  one  running  on  the  sequential  processor  (SP) 
and  the  other  on  the  associative  processor  (AP) . There  Is  also  an  Inter- 
face module  to  manage  the  Inter-processor  communications  via  the  external 
function  logic  and  shared  memory  (which  is  actually  AP  control  memory  that 
is  accessible  to  SF). 

The  system  attempts  to  overlap  as  much  as  possible  of  the  SP  and  AP 
execution.  The  SP  module  fetches  the  object  code,  decodes  it,  performs  the 
necessary  houskeeplng,  and  makes  a call  to  AP  for  an  array  operation. 

Willie  AP  is  staisfying  this  request,  SP  proceeds  to  the  next  object  code. 


SECTION  2:  HOW  TO  USE  APL-STARAN 


APL-STARAN  Is  a dialect  of  a subset  of  APL.SV.  Section  2 shows  how 
to  submit  an  APL-STARAN  job,  and  sections  3 and  4 explain  how  to  program 
In  APL-STARAN, 

This  manual  Is  based  on  Version  1.1  of  APL-STARAN  and  Is  Intended  for 
use  with  the  RADCAP  Staran  computer.  Access  to  this  facility  Is  through 
the  RADC  Multlcs  system,  and  jobs  submitted  In  this  fashion  do  not  permit 
APL-STARAN  to  be  used  Interactively.  It  Is  necessary  to  first  prepare  a 
complete  APL  "terminal  session"  by  entering  each  APL  statement  as  a separate 
line  In  a Multlcs  segment  named  APL. IN.  This  segment  may  be  prepared  using 
the  Multlcs  editor  "edm",  or  other  equivalent  editor.  When  this  Is  completed 
the  job  is  sent  to  STARAN  by  using  the  Multlcs  command  "staran"  to  enqueue 
the  following  dos-lnput  (-di)  segment: 

$J0B  C "your  own  UIC"  3 

$RUN  SUM 

ilfWA 

#X 

#LD  DK1:SARREX  /NG 
#TSCM 

$RUN  DKlsAPL 
$FINISH 

The  APL  output  generated  by  STARAN  will  be  found  in  the  Multlcs  segment 
APL. OUT,  where  each  APL  input  statement  will  be  retyped  with  an  Indentation 
of  6 spaces,  followed  by  the  appropriate  APL  response.  A sample  of  each 
step  of  this  procedure  may  be  found  on  the  next  page. 


NOTE:  In  all  the  examples  of  actual  terminal  input/output  that  follow,  the 
terminal  used  required  that  the  symbols  C»  1 » and  ^ be  entered  and  received 
as  and  ~i  respectively. 
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edm  apl . dl 
Se^nent  not  found, 

I nput . 

^dOB  i<64,46^> 

♦RUN  SDM 
^#WA 

i#LD  DK1:SARREX/NG 
<^#TSCM 

$RUN  OKI: APL 
$FINISH 

Edit. 

w 

q 

r 2120  0.397  6.0G4  191 

edn  APL. IN 
Segment  not  found. 

Input. 

3 + 5 

4 Tl  6 
7-2 

Edit. 

w 

q 

r 2121  0.26R  1.026  54 
staran  -d I apl .dl 

1 request  submitted,  2 already  queued, 
r 2121  0.392  10.374  108 


from  Staran. SysDaemon:  staran  request  processed;  apl.dl 

pr  APL. OUT  1 
APL-STARAM  VI. 1 
08-OCT-75 
22:14:43 

3+5 

8 .noonor.oo 

4 Tl  6 
2 .40000E01 
7-2 
5.00000E00 


r 2259  0.150  3., 684  47 
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SECTION  3:  THE  BASIC  APL-STARAN  STATEIfliNTS 


Tliere  are  two  main  categories  of  mathematical  functions  in  APL-STARAN: 

(1)  A fundamental  set  of  basic  arithmetic  and  logic  operations,  the 
so-called  Primitive  Scalar  Functions,  and 

(2)  A set  of  array  manipulating  operations  known  as  the  Primitive  Mixed 
Functions. 

In  addition  to  these  two  main  categories  there  are  a number  of  speciallsced 
operations  which  are  not  conveniently  grouped.  Table  II  is  a concise  summary 
of  all  these  operations,  together  with  the  symbols  and  mnemonics  used  to  denote 
these  functions. 

There  are  also  two  modes  of  system  operation  for  APL-STARAN: 

(1)  Execution  Mode  in  which  each  APL  statement  is  compiled  and  executed 
immediately  after  being  read  from  the  system  input  file  (APL. IN) 

The  result  of  this  statement  execution  is  then  output  to  the 

APL. OUT  file  before  the  next  input  statement  is  read. 

(2)  Function  Definition  Node  in  which  a number  of  statements  are  successive 
ly  read  in, then  compiled  and  stored  for  later  execution. 

The  execution  mode  is  the  normal  mode  in  the  sense  that  the  system  is  initially 
in  this  mode  and  remains  in  it  until  the  function  delimiter  (DL)  is  encountered 
in  the  system  input.  This  section  of  the  manual  deals  with  the  execution  mode. 
Section  4 explains  the  function  definition  mode  and  the  procedures  for  entering 
and  leaving  that  mode. 

Basic  data  formats.  There  are  two  basic  data  types,  namely,  character  and 
numeric.  The  logical  data  t>T)e  is  a subset  of  the  numeric  data  type  (1  = "true" 
and  0 = "false").  If  a primitive  function  requires  logical  data  for  its 
operands,  the  system  will  automatically  convert  to  a 1,  any  argument  v;hose 
absolute  value  is  greater  than  or  equal  to  0.5.  If  a primitive  function  requires 
numeric  data  and  is  supplied  instead  with  character  data,  the  system  will 
automatically  convert  the  character  data  to  their  ASCII  code  values. 

Numeric  data  may  be  entered  in  the  e>:poncntial  format,  e.g.,  1.23E6  may 
be  entered  in  place  of  1230000.  Regardless  of  how  the  data  is  entered,  only 
the  six  most  significant  digits  of  any  number  will  be  displayed  on  output,  and 
the  internal  fortaat  used  for  the  storage  of  numeric  data  will  maintaivi  only 
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2A  bits  of  significance.  In  fact,  this  is  rounded  to  21  bits  for  all  relational 
tests.  The  absolute  value  of  any  exponent  may  not  exceed  38. 

Negative  numbers  are  denoted  by  prefixing  them  with  the  ASCII  circumflex 
symbol,  or  the  PL/1  logical  NOT  (^)  if  a 2741  terminal  is  being  used.  Vectors 
of  both  the  numeric  and  character  data  types  may  also  be  entered.  Numeric 
vectors  are  formed  by  simply  juxtaposing  their  components,  using  spaces  to 
separate  each  of  the  elements.  Character  vectors  are  entered  by  enclosing  the 
character  string  in  quotes  (').  Tlie  quote  symbol  may  also  be  embedded  within 
such  a character  string  by  using  two  adjacent  quotes. 

Primitive  Scalar  Functions.  The  basic  arithmetic  and  logic  functions  nay  be 
divided  into  two  classes: 

(1)  Dyadic  functions  which  require  two  arguments,  one  placed  on  each  side 
of  the  symbol  used  to  denote  the  function,  and 

(2)  Itonadic  functions  which  have  only  one  argument,  placed  to  the  right  of 
the  function  symbol. 

In  APL-STAPvAN  there  are  19  dyadic  and  9 monadic  functions  in  the  primitve  scalar 
category.  These  are  all  listed  in  Table  II.  The  meaning  of  each  function  should 
be  reasonably  well  kno:m,  or  may  be  determined  by  experimental  use.  If  a 
precise  definition  is  desired,  any  standard  APL  text  may  be  consulted. 

Although  these  functions  are  defined  in  terms  of  operating  on  scalar 
arguments,  they  may  also  be  used  with  vectors,  or  even  higher  dim.ensional  arrays. 
If  the  function  is  monadic^  the  effect  of  having  an  array  as  an  argument  would 
be  to  produce  a I'esult  of  the  same  size  and  shape  as  this  argument.  Each 
element  of  the  result  would  be  obtained  by  applying  that  parti.cular  function  to 
the  con espondlng  element  of  the  argument. 

For  the  dyadic  functions  there  are  two  distinct  situations  which  can 
occur.  The  determining  factor  is  whether  the  left  and  right  argument  arrays 
are  confojmable  or  not  (that  is,  of  the  same  size  and  shape).  If  both  argument 
arrays  are  conform'ble,  then  the  result  will  conform  to  this  size  and  shape. 

Its  elements  will  be  the  result  of  applying  this  function  to  the  argument 
arrays  on  an  elcment-by-elcroent  basis  in  a manner  similar  to  the  monadic  case. 

If  the  arguments  are  not  conformable,  then  there  are  two  further  cases 
whicl)  can  occur.  The  first  case  arises  when  one  of  the  arguments  is  a scalar 
or  a single  element  array.  In  this  case  it  will  be  "extended"  to  be  conformable 


! 
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with  the  other  argument,  and  the  result  will  be  generated  as  if  both  arguments 
had  been  conformable.  If  neither  argument  is  a single  element,  then  a LENGTH 
or  RANK  error  occurs.  This  will  not  be  detected  by  the  APL-STARAN  system. 

Rather,  the  left  argument  will  be  forced  to  be  conformable  with  the  right 
argument,  using  "garbage"  from  the  workspace,  if  necessary,  to  fill  out  the 
left  argument.  The  result  will  thus  be  conformable  with  the  right  argument. 

Assignment  Statements.  Except  for  "branching"  statements  (which  occur  in 
defined  functions,  and  will  not  be  discussed  until  Section  4),  all  statements 
produce  a result  of  some  sort  which  is  generally  written  into  the  APL.OUT 
file.  Instead  of  outputting  this  result,  however,  it  may  be  stored  in  the 
workspace  for  future  calculations.  The  symbol  to  denote  the  storing  operation 
is  IS,  and  statements  of  this  type  are  usually  referred  to  as  "specification" 
or  "assignment"  statements;  they  specify  a variable  name  to  hold  the  result 
generated.  To  use  such  stored  quantities  one  merely  uses  the  variable  name  which 
is  "holding"  them,  whether  they  be  single  numbers,  or  vectors,  or  general 
arrays.  The  rule  for  forming  variable  names  is  that  they  must  begin  with  the 
letters  VA  followed  by  a sequence  of  1 to  3 other  letters  or  numbers. 

Consider  the  following  example  of  an  APL.OUT  file. 

VAX  IS  4+5 
VAX 

q.oooooF.oo 

2+VAX 

i.ioonoEoi 

VAY  IS  2 3 -5 
•'2  Tl  VAY 

•'ll  .onnooF.oo  •’s.nooooEoo  i.oooooeoi 

VAY 

2.oooooEno  3.nooooEon  ■'s.oooooeoo 

Line  (1)  illustrates  the  assignment  operation.  The  indentation  in  line  (2) 
indicates  that  there  was  no  output  as  the  result  of  performing  the  calculation 
in  line  (1);  the  system  was  waiting  for  another  input.  Calling  for  the  variable 
in  line  (2)  outputs  its  expected  value  in  line  (3).  The  variable  may  also  be 
used  in  calculations  as  shown  In  lines  (4)  and  (5) . Lines  (6)  through  (10) 
illustrate  the  storing  of  a vector  along  with  the  "extension"  of  a scalar  to 
be  conformable  with  a 3 clement  vector  as  required  by  the  times  (raultlplicatiop) 
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function.  ! 

The  general  rule  on  when  output  occurs  and  when  it  does  not,  is  that  It  i 

always  occurs  except  when  the  operation  leftmost  in  the  statement  is  the  jj 

assignment  function.  The  only  exception  to  this  is  with  the  use  of  the  output  j 

function.  A detailed  discussion  of  this  function  will  appear  below.  It  j 

is  also  worth  observing  that  if  any  quantity  (either  a number  or  a literal  1 1 

string)  is  entered  alone  as  an  entire  APL  statement  in  itself,  it  becomes 
the  result  value  generated  by  the  statement,  and  as  such  will  be  output  to  ii 

I- 

the  APL. OUT  file,  as  shown  in  the  following  example. 

2? 

2.30000^01 

’ALPHARETIC  DATA’ 

ALPHABETIC  DATA 

It  must  also  be  noted  that  any  number  of  operations  may  be  specified 
in  one  APL  statement.  This  includes  assignment  and  I/O  operations  as  well 
as  regular  mathematical  operations.  If  more  than  one  operation  is  specified 
in  a given  statement,  the  order  of  operations  is  strictly  from  right  to  left 
unless  parentheses  are  used  to  group  operations  into  some  other  sequence. 

This  is  an  important  point  since  it  differs  from  other  programming  languages 
which  generally  employ  a more  complicated  order  (usually  involving  some 
hierarchy  of  operations)  and  working  from  left  to  right.  In  APL  all  opera- 
tions are  equal  in  determining  the  order  of  execution.  In  the  absence  of 
parentheses,  the  first  operation  to  be  performed  is  the  rightmost.  If  this 
is  not  the  intended  order  of  execution,  parentheses  may  be  used  freely 
(including  parentheses  within  parentheses)  to  group  the  operations  as 
desired.  If  parentheses  are  used,  they  must  be  used  in  pairs,  and  every- 
thing within  a given  pair  will  be  evaluated  prior  to  its  being  used  as  a 
quantity  in  some  other  operation  external  to  that  pair.  ITie  follov;ing 
output  example  illustrates  these  points. 


1+2+3+4 

i.onoooF.oi 

(2  Tl  3)+(4  Tl  5) 

2.60000E01 

(5-2)-G 

^S.OOOOOFOO 

5-2-6 

9.00000E00 

The  only  line  that  should  require  explanation  is  line  (7).  It  is 
important  to  remember  the  rifiht  to  left  rule.  Do  not  read  this  line  as  5 
svibtract  2 gives  3,  subtract  6 gives"*!.  That's  the  way  to  read  line  (5). 
Because  of  the  right  to  left  rule,  line  (7)  should  be  read  as  2 subtract  6 
gives'’^,  and  5 subtract"'^  gives  9.  Another  way  of  looking  at  it  is  that 
every  symbol  operates  on  everything  to  the  right  of  it,  up  to  the  first 
parenthesis  which  encloses  it  (assuming  it's  enclosed  in  parentheses). 

Primiti.ve  Mixed  Functions.  Arrays  arc  classed  in  accordance  with  their 
dimensionality.  A vector  is  a one-dimensional  array,  as  opposed  to  a matrix: 
whicli  is  a two-dimensional  array.  In  accordance  with  this  classification 
a scalar  should  be  considered  as  an  array  of  zero -dimens ion.  The  dimension- 
ality of  an  array  is  usually  called  its  rank  or  shape. 

The  elements  of  an  array  are  the  individual  ninnbers  in  a numerical 
array,  or  each  separate  character  in  a literal  array.  These  elements  are 
identified  or  distinguished  from  each  other  by  their  indices . An  index  is 
a number  which  locates  the  element  relative  to  the  origin  or  beginning  of  the 
array.  For  instance,  in  a vector  a single  index  for  each  element  suffices 
to  locate  it  relative  to  the  first  element  in  the  vector.  The  first  element 
may  be  given  the  index  0 or  1 depending  on  the  index  origin  setting,  as 
desciibed  under  "system  dependent  functions".  A matrix  v’ould  require  two 
indices  for  each  element;  one  to  identify  the  row  in  wliich  it  appears,  the 
other  to  Identify  the  column.  Rows  and  colu.nns  are  numbered  startling  at 
either  0 or  1 depending  on  the  index  origin.  Higher  dimensional  arrays 
arc  allowed,  and  are  indexed  similarly. 

The  size  of  a specific  array  is  given  by  the  number  of  elements  in  the 
array.  (The  size  of  a matrix  would  require  t\,’o  numbers  to  specify  it.)  As 
an  example,  the  size  of  the  vector  1 7 9 3 is  h,  while  the  size  of  the 
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matrix 

2 5 14 

13  6 

would  be  2 3 (the  row  Index  is  always  given  first) > It  is  also  possible 

to  have  a vector  of  size  0.  This  means  there  are  no  elements  in  the  array, 
and  it  is  referred  to  as  an  emp ty  vector. 

The  Reshape  (or  Restructure)  operation  is  a dyadic  function  which  re- 
structures the  right  operand  (which  can  be  any  array)  to  have  the  size  and 
shape  specified  by  the  left  operand.  This  operation  may  be  used  to  make 
arrays  of  any  size  or  shape.  If  there  are  not  enough  elements  in  the 
original  array,  they  are  used  over  again,  starting  from  the  beginning. 

If  there  are  too  many  elements,  the  unused  ones  are  dropped.  Example: 


VAX  IS  R RM  2 
VAX 

2.onoonF.no  2.ooonnEno 

2.r)OoooEno  2.00000E00 

VAY  IS  S RH  1 2 3 4 
VAY 

l.OOOOOEOO  2.00000E00 

i.noonoEno  2.onoooEoo 

VA7.  IS  3 RH  1 2 3 4 5 
VAZ 

l.OOOOOEOO  Z.nOOOOEOO 

3 4 RH  VAZ 

i.nnonoEoo  z.onnooEoo 

2.oonnnEoo  3.00000E00 

5.ononnEno  i.nooooEoo 

2 1 RH  VAY 
i.nnnooEOO 
2 .onoooEno 


2.noonoEf)0 


3.0000OE00 

3.00000E00 


6 


3.n0000E00 

3.noOOOEOO 

1. nonnoEOo 

2. nonooEOo 


2.00000E00 


4.00000E00 

4.00000EO0 


l.OOOOOEOO 

2.00000E00 

3.00000EOO 


The  size  operation  is  a monadic  function  which  yields  the  size  of  the 
operand  (any  array)  on  its  right.  The  size,  operation  always  produces  a 
vector  for  its  result.  The  size  of  a single  number  is  an  empty  vector, 
whereas  the  size  of  a vector  is  Itself  a vector  of  size  1.  Thus,  the  size 
operation  applied  to  the  size  of  any  array  A (i.e.,  the  size  of  the  size  of 
A)  yields  the  shape,  or  rank,  of  the  arraj’,  (i.e.,  its  dimensionality) . 
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Exnriple: 


VAX  IS 

2 4 RH  2 5 *'9 

'! 

] 

VAX 

i 

2 .oononEOO 

S.nOOOOEOO  •'9.00000E00 

2.00000EOO  , 

5 .OOOfJOEOO 
Rll  VAX 

"fl.OOOOOEOO  2.00000E00 

5.00000E00  ! 

2 .OOnOOECJO 

4.00000EOO 

! 

Rll  1 2 
i.inoooEOi 

345654321 

1 

f 

I 

f 

I 


i 


! 

f 

I 


The  Indcxlnf;  operation  is  somewhat  special  in  the  way  it  handles  Its 
operands.  The  function  performed  by  the  operation  is  the  selection  of 
specific  elements  in  any  array  by  the  specification  of  the  indices  of  the 
selected  elements.  The  notation  for  indexinB  is  the  enclosure  v/ithin 
brackets  of  the  desired  indices,  which  may  themselves  be  in  the  form  of  an 
array.  The  brackets  must  be  placed  immediately  to  the  right  of  the  array 
to  be  indexed.  Two  indices  are  needed  to  index  a matrix,  and  they  are 
separated  from  each  other  by  a semi-colon.  If  either  one  of  the  indices 
is  omitted  from  a matrix  index,  the  entire  row  or  column  specified  by  the  j 

other  index  is  assumed  to  be  referenced.  Tlie  case  of  higher  dimensional  I 

arrays  is  handled  analogously.  Example:  | 

i 

i 

c 

4.00000E00  ] 

8.00000E00  1 


8.00000E00 

8.00000E00 

4.0n000E00 

8.00000EOO 

i 
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VAX  IS24RH  12345678 
''AX 

i.oorii;OLoo  2.nnoooEoo  3.nooonEOO 

5.00000EOO  G.OOOnOEOO  7.00000E00 

VAX(i<l;l(i> 

i.nnoooEOO 

VAX<:<2;3(^> 

7.000r;';[;0f) 

B.oonnoEoo  B.nnoooEnn 

3.rnobOEf)o  7.nonnoEOo 

VAX(^<2  1 2;3  3 1 4<::> 

T.onoonron  i.nonooEOo 

5.onr)onEOO  3.nfjoooEno 

7.cnoooi:oo  t.oooooeoo 


7.00000E00 


5.nf)nnnr:oo 

i.nnnoni:no 

5.00000E00 


The  Index  Genera Izor  Is  a monadic  function  requiring  a positive  integer 
for  Its  operand.  It  generates  a vector  of  successive  integers  of  the  size 
specified  by  the  operand.  If  the  operand  is  zero,  it  generates  an  empty 
vector.  The  first  integer  In  the  vector  will  be  the  index  origin.  Example: 

10  12 

i.ooonoEOO  2.ononoE9n  s.onoooEoo  4.00000EOO 
5.00000EOO  G.nnoooEoo  7.00000E00  s.oooooeoo 

n.ononoEno  i.oooooEOi  i.ioonoEOi  1.20000E01 

10  0 

The  Least-Index-Of  function  (dyadic  10)  requires  a vector  for  its 
left  argument  but  will  accept  an  array  of  any  size  or  shape  for  its  right 
argument.  It  generates  a result  array  which  is  conformable  with  the  right 
argument.  Each  element  in  the  result  is  the  lowest  index  of  the  left 
argument  where  the  corresponding  element  of  the  right  argument  is  to  be 
found.  Any  elements  in  the  right  argument  which  do  not  occur  in  left 
argument  will  generate  a least  index  which  is  one  greater  than  the  highest 
index  of  the  left  argument.  Example: 

31425  1027 
4.0OOOOEOO  6.0Q000E00 

The  Ravel  function  (monadic  comma)  manipulates  the  size  and  shape  of 
any  argument  to  make  it  into  a vector  by  "stringing  out"  the  elements  in 
row-major  order.  In  its  dyadic  form  the  comma  denotes  the  catenate  function. 
For  this  function  the  arguments  must  be  either  scalars  or  vectors.  The 
function  then  creates  a longer  vector  by  appending  the  right  argument  to 
the  right  end  of  the  left  argument.  If  the  argmnents  are  higher  dimensional 
arrays  they  should  be  conformable  (or  one  should  be  a scalar  or  single 
element  array).  The  right  argument  will  then  be  appended  to  left  argument 
along  the  last  dimension.  For  instance,  in  the  case  of  two  matrices,  the 
result  array  will  have  the  same  number  of  rows  as  the  arguments,  but  the 
number  of  columns  (i.e.,  last  dimension)  will  be  equal  to  the  sum  of  the 
number  of  columns  in  the  left  and  right  arguments.  Tlie  columns  on  the 
right  side  of  the  result  will  be  those  from  the  right  argument.  The  sit- 
uation for  other  higlier  dimensional  arrays  will  be  analogous.  If  the 
arguments  are  not  conforiiiable,  they  will  be  made  conformable  as  described 
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under  "Primitive  Scalar  Functions".  Examples  will  be  found  following  the 
discussion  below. 

This  function  also  has  an  "indexed"  form  created  by  enclosing  an  integer 
indcix  in  brackets  and  inserting  it  betv7een  the  comma  and  right  argument. 

The  function  of  the  index  is  to  select  the  dimension  along  which  the  cate- 


nation should  take  place.  Legal  index  values  are  thus  limited  to  the 


number  of  dimensions 

in  the  arguments  of  the  function.  If  the  index  is  not 

an  integer  the  function  becomes  the 

: laminate  function.  Lamination  creates 

an  array  of  rank  one 

higher  than  the  arguments  by  inserting  a new  dimension 

of  size  2 above  the 

dimension  indicated  by  the  floor  of  the  fractional 

function  index.  The  procedure  for 

forming  the  result  is  best  illustrated 

through  the  various 

examples  given 

below. 

2 1,3  4 

2 .ooonofino 

l.nOOOOFOO 

3. OOOOOEOO  4. OOOOOEOO 

9,2  2RH 

10  4 

9.00000EOO 

1. OOOOOEOO 

2. OOOOOEOO 

9 .onooornn 

3.00000E00 

4. OOOOOEOO 

VAX  IS 

2 3 R!I  10  6 

VAX,(i<l<t>9 

i.nnoooF.no 

2 .OOOOOEOO 

3. OOOOOEOO 

4 .9(?00f)h00 

s.ooonoEno 

6. OOOOOEOO 

9 .onoooF.oo 

9. OOOOOEOO 

9. OOOOOEOO 

1. 90ooor.no 

2 .nonnoEoo 

3. OOOOOEOO  9. OOOOOEOO 

4.ononoF;oo 

5. OOOOOEOO 

6. OOOOOEOO  9. OOOOOEOO 

9,i<2.5(:;>VAX 

9 .OOOOOF.nO 

i.oonnoEoo 

9 .OOnOOFOO 

2. OOOOOEOO 

9. 90000EOO 

3. OOOOOEOO 

9 . oonoor 90 

4.oononEoo 

9 .nonociFOO 

s.nnoocEoo 

9 .ooonor.nn 

6. OOOOOEOO 

9,  <:<1.5 

(;:>VAX 

n.oooooroo 

9. OOOOOEOO 

9. OOOOOEOO 

1 .OOOOOLOO 

2.nnoooEno 

3. OOOOOEOO 

. o'''ioono 

9. OOOOOEOO 

9. OOOOOEOO 

or  90 

5.000001:00 

6. OOOOOEOO 

The  Reducllnn  operator  (/)  in  APL-STARAN  requires  a vector  for  its 
right  (and  only)  "real"  argument,  but  it  also  requires  one  of  the  C functions 
+,  TI,  MX,  MN,  AM,  OR  as  a "pseudo"  left  argument.  It  generates  a scalar 
which  is  the  result  that  would  be  obtained  if  the  specified  function  were 
inserted  between  each  of  the  elements  of  the  vector.  In  other  works  "+/VAX"  I 

yields  the;  sum  of  all  the  elements  in  the  vector  named  VAX. 

The  Execute  function  (EX)  requires  a character  vector  as  its  right 
argument  and  executes  (evaluates)  the  character  string  as  if  it  had  been  , ; 

entered  through  system  input  APL.IN.  It  works  by  generating  an  Internal 
call  to  the  /d^L-STARAN  compiler,  and  produces  the  same  results  (including 
user  function  definition)  as  would  normally  be  obtained  from  the  compiler. 

System  Dependent  Functions.  The  I-beam  primitive  (IB)  may  be  used  monadic- 
ally  to  accomplish  a number  of  system  oriented  functions.  In  all  cases  the 
argument  must  be  either  a scalar  or  a two  element  vector.  Tlie  allowed 
scalar  arguments  and  their  respective  system  functions  are; 

IB  21  returns  the  elapsed  CPU  time  in  1/60  seconds. 

IB  48  returns  the  index  origin. 

IB  50  returns  the  digits  control  setting  (number  of  digits  used  for  output). 

IB  51  returns  the  width  control  setting  (number  of  output  characters  per 
line) . 

IB  64  terminates  APL-STARAN. 

The  use  of  a two  clement  vector  for  the  argument  allows  a number  of 
system  controls  to  be  given  values.  In  each  case  the  first  element  of  the 
vector  determines  the  precise  system  function  to  be  performed,  and  the 
second  clement  is  the  data  necessary  to  accomplish  it.  (A  more  complete 
discussion  of  the  I/O  opcirations  will  be  found  in  the  next  section)  . Tlie 
implemented  function  are: 

IB  36  H (1  5 N ^ 7)  open  dataset  "N"  for  I/O. 

IB  37  N (1  5^  N S 7)  closes  dataset  "N"  for  I/O. 

IB  48  N (0  < N < 1)  sets  index  origin  to  N. 

IB  50  N (1  < N S 6)  sets  digits  control  to  N. 

IB  51  N (32  < N)  sets  width  control  to  N. 

Input /Output  Functions.  If  the  output  of  a vector  will  not  fit  on  one  line, 
subsequent  lines  wi].l  be  used  and  be  indented  to  indicate  the  continuation 
of  the  vector.  An  empty  vector  is  output  as  a blank  line.  In  matrix 
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output  the  columns  will  be  aligned,  provided  each  row  fits  on  a single  line. 

If  a row  requires  more  than  one  line,  subsequent  lines  will  be  indented  to 
indicate  the  continuation  of  the  row.  Column  alignment  will  hold  for 
corresponding  lines  of  each  row. 

APL-STARAN  uses  8 datasets  for  input  and  8 for  output.  For  purposes  of 
communicating  with  STARAN's  operating  system  these  datasets  are  named 

DIfi,  Dll,  ...,  DI7  for  input,  and  DO0,  DOl D07  for  output.  The 

system  input  is  dataset  DI0  and  is  Internally  assigned  to  a Multics  segment 
named  APL.IN.  The  system  output  is  dataset  DO0  which  is  assigned  to  a 
Multics  segment  APL.OUT.  Both  of  these  assignments  may  be  altered  by  using 
the  DOS-11  monitor  c_.nnnand  ^ASSIGN  before  executing  the  DOS  command  0RUN 
DKlrAPL.  The  details  of  this  procedure  are  described  in  Digital  Equipment 
Corporation's  Manual  "Disk  Operating  System  Monitor  Programmer's  Handbook" 
(DEC-ll-OMONA-A-D) . The  other  datasets  are  internally  assigned  in  input/ 
output  pairs  to  a corresponding  Multics  segment  named  APLN.DAT  where  N is 
a number  from  1 to  7 that  matches  the  dataset  number.  For  example,  DI2  and 
P02  are  both  assigned  to  the  Multics  segment  APL2.DAT.  These  assignments 
may  likewise  be  changed  by  the  DOS  command  0ASSIGH. 

Tlie  system  input  and  output  datasets  are  automatically  opened  and  closed 
by  the  initiation  and  termination  of  APL-STARAN.  The  other  datasets  must 
be  opened  by  the  appropriate  I-Beam  function  before  they  can  be  used  for 
input  or  output.  Tlie  number  of  datasets  that  may  be  simultaneously  open 
is  limited  by  the  available  buffer  space.  In  order  not  to  lose  any  data 
that  has  been  written  into  these  datasets  they  must  be  closed  witVi  the 
appropriate  I-Beam  before  terminating  APL-STARAN.  When  datasets  1 through 
7 are  written,  the  new  data  is  appended  to  what  is  already  in  the  file.  In 
the  case  of  dataset  0,  the  file  APL.OUT  is  erased  and  rewritten  each  time 
APL-STARAN  is  initiated. 

The  monadic  function  IN  uses  its  argument  (between  0 and  7)  to  identify’ 
the  dataset  frora  which  to  read.  Character  data  is  read  serially  from  this 
dataset,  starting  from  the  present  read  "position"  up  to  the  first  terminator 
encountered  (e.g.  carriage  return).  The  new  read  position  is  set  following 
this  terminator.  The  character  vector  thus  read  becomes  the  value  returned  by 
this  function.  There  is  a ni ladle  form  of  this  function  (niladJc  means  ^ 
arguments)  which  is  equivalent  to  IN  0.  In  other  words,  Nl  has  no  argument 
and  reads  from  the  system  input  file  APL.IN. 
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There  are  two  forms  of  the  output  function.  Bare  output  (OB) 
differs  from  terminated  output  (OT)  in  that  the  latter  function  supplies  a 
terminator  (new— line  character)  to  the  end  of  the  output  string.  The 
left  argument  identifies  the  dataset  v;here  the  output  is  to  go.  If  it  is 
t it  may  be  omitted,  which  is  equivalent  to  the  function  becom5.ng  a monadic 
function.  In  both  cases  the  right  argument  is  the  data  to  be  output. 


SECTION  4:  DEFINING  ATL-STAMN  FUNCTIONS 


The  system  supplies  a function  definition  operation  (DL)  whicli  enables 
the  user  to  orj>nnize  any  specific  sequence  of  basic  arithmetic  and/or  array 
processing  operation?!  into  a defined  function.  This  defined  function  will 


be  identified  by  a function  name  V7hich  must  be  formed  as  described  hclcn-;. 
Once  the  function  is  defined  it  may  be  executed  l<y  using  its  name  in  an 
APL  statement. 

Defined  functionfj  may  be  divided  into  the  same  two  groups  used  to 
classify  the  fundamental  system-supplied  functions^  namely,  monadic  and 
dyadic.  In  addition  there  is  a third  classification  available  for  defined 
functions  called  niladic  - no  operands.  Functions  in  this  classification 
may  be  considered  as  stand-alone  programs  whlcli  employ  I/O  operations  to 
input  their  operands  and  output  their  results.  The  present  discussion 
begins  with  this  case.  The  procedure  for  defining  functions  with  explicit 
operands  and/or  explicit  results  is  discussed  in  a following  section. 


Function  definition.  Consider  the  following  system  input  as  an  example 
of  niladic  function  definition.  The  corresponding  output  sequence,  as  well 
as  a detailed  explanation,  follows  the  exampli*. 

Example  Csystem  input) : 


PL  NFAV 

'NFAV  CALCULATES  THE  AVERAGF.  OF  A SET  OF  HUnCERS.' 
VANUM  IS  EX  MI 

LAI  HP:  GO  LADO  IF  0=-RH  VATST  IS  Ml 
VAUIJM  IS  VAHLIM,  EX  VATST 
GO  LA  I HP 

LADO:  'THE  HUMRERS  ARE:' 

VANUM 

OS  'THE  MEAN  IS  ' 

+ /VANUM  nV  RH  VANUM  DL 
NFAV 
12  3 4 
5 11 

12  13  14  15 
IB  64 
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Example  continued  (system  output) : 

DL  MFAV 

'NFAV  CALCULATES  THE  AVERAGE  OF  A SET  OF  NUMBERS.' 

VANUM  IS  EX  Nl 

LAINP;  GO  LADO  IF  0=RH  VATST  IS  Nl 

VANUM  IS  VANUM,  EX  VATST 

GO  LA  IMP 

LADO:  'THE  NUMBERS  ARE:' 

VANUM 

OB  'THE  MEAN  IS  ' 

+/VANUM  DV  RH  VANUM  DL 
NFAV 

NFAV  CALCULATES  THE  AVERAGE  OF  A SET  OF  NUMBERS. 

THE  NUMBERS  ARE: 

i.onnncEOO  2.POonoEOO  3.00000E00  4.nooooEOO 
s.onoooEno  i.iooooEOi  1.20000E01  1.30000E01 

1.40000E01  1.50000E01 

THE  MEAN  IS  R.OOOOOEOO 
IB  G4 

The  first  line  switches  the  system  from  the  execution  mode  to  the 
function  definition  inode.  This  is  accomplished  by  entering  DL  followed 
by  the  name  tliat's  to  be  assigned  to  the  function.  This  Al^L  statement  is 
known  as  the  header  line  of  the  function.  The  names  of  niladic  functions 
must  begin  with  NF  and  be  followed  by  from  one  to  three  other  alphanumeric 
characters.  In  the  example  above  the  function  is  named  NFAV.  An  explanation 
of  what  the  program  does  is  provided  by  the  message  that  is  the  first  line 
in  the  body  of  the  function  definition. 

Statements  entered  when  the  system  is  in  the  function  definition  mode 
are  not  executed  at  the  time  of  entry.  They  are  merely  checked  for  syntax 
errors  and  then  compiled  and  stored  for  later  execution.  The  function 
definition  mode  is  ended  by  entering  DL  after  the  last  statement  (either 
appended  to  the  last  statement,  as  in  the  example,  or  entered  alone  on  the 
line  following  tlie  last  statement).  When  the  function  definition  mode  ends, 
the  system  returns  to  the  execution  mode.  The  entry  following  the  function 
definition  in  the  example  above  shows  how  to  "execute"  the  function  (by  using 
its  name  alone  on  a line) , and  the  succeeding  lines  show  a sample  "run"  of 
the  program. 


I 
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In  tlxe  normal  courm  of  events  the  execution  of  a defined 
function  means  the  execution  of  the  statw.ueuts  in  the  body  of  the  function 
taken  in  order,  starting  with  the  first.  This  order  may  he  altered  by  the 
"branch"  function  GO.  This  primitive  function  must  have  as  its  right 
argument  the  label  of  the  line  to  which  control  is  to  be  transferred. 

Labels  must  begin  with  lA  and  be  followed  by  from  one  to  three  other 
alphanumeric  characters,  Tliey  must  be  the  leftmost  entry  in  the  lines 
that  arc  possible  targets  of  the  brand;  Insti nction,  and  they  arc  separated 
by  a colon  from  the  lines  they  arc  labelling.  A branch  to  the  "null" 
label  (LA  alone,  without  any  other  characters)  causes  an  exit  from  the 
program.  This  will  of  course  also  occur  when  the  last  line  of  the  program 
is  executed,  provided  it  is  not  a branch. 

The  branch  may  be  made  conditional  if  the  target  label  of  the  GO 
Instruction  is  also  used  as  the  left  argument  of  the  primitive  function  IF, 
IF,  in  turn,  must  have  as  its  right  argument  an  APL  expression  that  will 
evaluate  to  either  "true"  or  "false",  eg.  1 or  0.  In  the  example  above, 
there  are  instances  of  both  the  conditional  and  unconditional  branches. 

It  is  also  possible  to  branch  to  one  of  a number  of  target  statements 
by  providing  the  GO  function  with  an  indexed  vector  of  labels.  In  this 
case  the  index  selects  one  specific  label  (unless  the  index  is  out  of  range, 
in  which  case  control  is  transferred  to  the  statement  following  the  branch). 
An  example  would  be 

GO  LAGO  lA  LAIN  LA  L/vOlU  (VAXj 


where  the  variable  V/JC  selects  one  of  the  five  labels  supplied. 

Note;  In  APL-STAIIAIJ  labels  linve  no  numeric  value  and  may  not  be  used 
as  arguments  for  any  primitive  functions  otlier  than  GO  and  IF, 


Local  and  p.lobal  variables.  In  the  preceding  ex£imple  it  is  entirely 
possible  that  the  variable  VATST  had  been  previously  specified  in  the 
workspace  and  was  holding  some  value  that  had  to  be  used  at  a later  time. 
In  executing  the  function  NFAV,  however,  its  value  was  respecified,  and 
after  the  function  execution  was  comnleted  VATST  would  be  an  empty  vector, 
and  its  original  value  would  have  been  lost.  To  prevent  this  from 
happening  VATST  may  be  specified  as  being  local  to  the  function.  However, 
tJiis  will  mean  that  during  the  execution  of  the  function  NFAV  the  global 
variable  VATST  v’ill  be  unavailable  for  use.  This  will  also  be  true  of 
any  functions  Invoked  by  NFAV.  ( In  this  particular  example  there  are 
none,  but  there  could  have  been). 

The  rule  on  local  and  global  variables  is  that  the  system  treats  all 
variables  as  global  unless  they  are  made  local  to  a function  by  appending 
their  name,  preceded  by  a semi-colon,  to  the  name  of  the  function  when 
it  is  used  in  the  header  line  of  the  function  definition  process.  It  is 
possible  to  make  any  number  of  variables  local  to  a function  by  appending 
each  successively  to  the  header  line  of  the  function  during  the  definition 
process.  In  each  case  they  must  be  preceded  by  a semi-colon. 

The  first  example  below  shows  the  effects  on  the  variable  VAX  when 
it  is  left  global  and  has  its  value  respecified  within  a function.  The 
second  example  shows  how  to  save  a global  value  by  making  the  variable 
local  to  a function. 


VAX  IS  2 
PL  MFEXl 
VAX  IS  3 DL 
NF[-:X1 
VAX 

3.00000E00 

DL  NFEX2;VAX 

OB  'WITHIN  THE  FUNCTION  VAX  IS  ' 

OT  VAX  IS  4 DL 
NFEX2 

WITHIN  THE  FUNCTION  VAX  IS  4.00000E00 
VAX 

3.oonooF.oo 
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Function!^  with  exp1.:I.clt  opcirarulf?  and/or  results.  It  Is  possible  to  define 
functions  with  either  one  or  two  operands  explicitly  spe(pified.  The  one 
operand  specification  is  done  by  having  a variable  name  specified  In  the 
function  header  line  on  the  right  side  of  the  function  name,  and  separated 
from  it  by  a space.  The  names  for  such  functions  must  begin  with  MF  and  be 
followed  by  from  one  to  three  other  alphamnneric  characters.  In  the  follow- 
ing example  VAX  is  an  explicit  operand  variable,  and  the  function  MFSQR 
calculates  the  square  root  of  this  operand, 

DL  MFSQR  VAX 
VAX  * .5  01. 

MFSQR  10  4 

l.OOOOOEOO  1.41421E00  J..73205E00  2.00000E00 

In  this  case  the  function  must  always  be  used  with  an  operar  ' on  its 
right,  or  it  will  produce  an  error  message.  Local  variables  may  also  be 
specified  following  the  right-hand  operand;  however,  the  operand  variable(s) 
and  the  result  variable  ( if  there  is  one  ) arc  always  local  and  should  not 
be  included  among  the  local  variable  specification. 

If  two  operands  are  specified  (dyadic  function j one  must  be  on  the 
left  and  one  on  the  right,  as  is  the  case  with  all  dyadic  functions.  Each 
operand  must  be  separated  from  che  function  name  by  spaces,  and  the  function 
name  must  begin  with  DF  rather  than  MF.  As  an  example,  consider  the 
following  function  which  calculates  the  area  of  a triangle  in  terms  of 
its  base  and  height. 

QL  VARAS  nFTRI  VAHGT 
.5  TI  VABAS  Tl  VAHGT  DL 
3 QFTRI  10 
l.SOOOOEfJl 
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It  is  also  possible  to  explicitly  specify  a result  variable  which 
is  always  local  to  the  function  involved,  but  permits  a value  to  be 
returned  as  a result  of  the  function  invocation.  This  permits  defined 
functions  to  be  eiribeded  within  APL  statements  in  much  the  same  way  that 
the  basic  arithmetic  operations  are  used.  A result  variable  is  explicitly 
specified  by  entering  its  name  followed  by  IS  on  the  left  of  the  function 
name  ( and  left-hand  operand  if  there  is  one  ).  For  example: 


ni  VARES  IS  MFSaR  VAX 
VARES  IS  VAX  * .5  0!. 

10  + 3 Tl  MFSQR  »» 
1.60000E01 


All  user  defined  functions  may  be  used  in  other  defined  functions 
Including  their  own  definition,  which  allows  recursive  function  definition. 

None  of  an  APL-STARAN  function  definition  is  compiled  until  the  last 
line  of  the  function  has  been  read  in.  Compilation  begins  at  this  point 
and  will  stop  when  the  first  syntax  error  is  encountered.  Tlie  line 
number  of  the  statement  containing  the  error  will  be  reported.  The  header 
line  of  the  function  is  considered  as  line  0.  If  the  error  pertains  to 
the  use  of  a label  in  a branch  statement  which  does  not  appear  as  a label 
on  any  line  in  the  program,  the  line  number  of  the  error  will  be  reported 
as  the  last  line  number  plus  one. 
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SECTION  5:  APL-STARAN  PERFORMANCE  MEASUREMENTS 


The  execution  times  for  various  basic  APL  statements  have  been 
measured  for  both  APL-STARAN  and  APL/370.  Tlie  APL/370  version  was  run  on 
the  IBM  Systcm/370,  Model  155  at  Syracuse  University  In  July  and  August  1975. 
During  these  tests  the  system  load  was  light  (between  4 and  6 users),  but  the 
370  data  will  nevertheless  show  some  small  effects  of  core-swapping  vrhich 
, will  not  be  present  In  the  STARAN  data.  In  both  cases  the  tests  were  run  by 

a timing  function  MFTIM  which  caused  1000  executions  of  the  test  statement 
(except  for  the  very  long  execution  times  which  used  a repetition  factor  of 
200).  The  MFTIM  function  is  shovm  below. 

DL  VAR  IS  M FT  111  VAC  NT;  VAT  I M 
VATIM  IS  IB  21 
LAGOrMFTST 

GO  LAGO  IF  0<VACNT  IS  VACNT-1 
VAR  IS  (IB  21)-VATIM  DL 
DL  NFTST 

The  function  NFTST  was  a one  line  niladic  function  which  contained 
the  teat  statement.  A test  was  first  run  with  NFTST  being  an  empty  function 
• (i.e.,  only  the  header  line),  and  the  time  for  this  execution  was  subtracted 

from  all  other  measurements  to  correct  for  the  overhead  involved  In  running 
MFTIM.  These  times  are  listed  In  the  columns  labelled  "Execution  Time" 
of  Table  III.  The  times  listed  arc  In  milJiseconds  and  represent  the  CPU 
time  required  for  a single  execution  of  the  test  statement. 

In  all  cases  the  results  of  the  various  primitive  functions  were 
assigned  to  variable  VAX  to  avoid  the  overhead  associated  with  result  print- 
out. Even  in  this  case  there  is  an  overhead  associated  with  the  assignment 
function.  To  correct  for  this  two  simple  assigr.ment  tests  were  run,  one  for 
a scalar  assignment  (test  1)  and  one  for  a 256  element  vector  assigiunent 
(test  4).  The  columns  labelled  "Adjusted  Times"  have  been  corrected  for  this 
effect.  Tliey  were  obtained  by  subtracting  the  assignment  overhead.  The  scalar 
overhead  (test  1)  was  used  to  correct  tests  2,  3,  11,  12,  16,  and  17.  The 
other  tests  were  adjusted  using  the  vector  overhead. 

Since  the  STAR/'N  implementation  uses  essentially  the  same  amount  of 
time  for  a scalar  as  for  a vector  or  higher  dimensional  array,  the  execution 
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time  improvement  of  the  STARAN  over  the  370  will  be  most  evident  for  larger 
size  vectors.  The  data  clearly  demonstrate  this,  yielding  a factor  of  4 or 
5 ."IS  the  average  Improveuiient  for  the  case  o-  a 256  element  vector. 

There  appear  to  be  two  anomalies.  First,  the  370  uses  an  excessively 
laige  amount  of  time  to  compute  the  AN  function  (tests  12  and  13),  but  this 
is  due  to  l.lie  need  for  a floating  point  conversion  which  is  not  required  in 
the  STARAN  implementation.  Tlie  second  anomaly  occurs  in  the  plus  reduction 
for  the  STARAN.  This  is  due  to  the  serial  execution  technique  which  is  used, 
and  will  also  be  true  for  the  times  reduction.  However  it  is  not  true  for 
the  AN,  OR,  MX,  or  MN  reductions  which  use  the  full  array  capability  of  the 
STARAN. 


13-30 


TABLE  III.  COMPARATIVE  PERFORMANCE  MEASUREMENTS 


SECTION  6:  APL-STARAN  IM'LEMENTATION  DETAILS 


The  AP  module  which  runs  the  arrays  Is  named  SARREX  (^t<aran  Army  Exec- 
ution) . It  is  written  in  APPLE  and  is  loaded  into  Staraii  by  the  Goodyear 
system  program  SDxi.  It  consists  of  a relocatable  section  which  is  loaded 
into  pages  0,  1,  and  2,  plus  a number  of  short  absolute  sections  to  be 
loaded  at  hexadecimal  addresses  610,  8200,  8300,  and  B08A.  There  Is  also 
some  residual  debugging  instrumentation  located  at  absolute  addresses  8400 
and  B030.  This  module  is  not  designed  to  run  under  control  of  SD'i  but  is  rather 
initialized  by  a PDP-11  subroutine  najned  INSTAR  and  then  runs  under  control 
of  a PDP-11  module  named  PARREX  (^DP  Array  ITxecution  Control). 

After  APL-STAlh\N  has  been  initialized  the  Al'  comes  to  a halt  by  executing 
a WAIT  instruction  imbedded  within  the  initialization  routine.  Subsequent 
calls  on  the  AP  occur  when  PARRliX  loads  the  necessary  parameters  into  an  argu- 
ment area  called  APARG  (AP  _^rjjument  area  - this  area  is  a block  of  AF  control 
memory  which  is  shared  with  the  PDP-11)  and  then  issues  an  EXF  (External  Func- 
tion Call)  to  start  the  AP.  PARREX  then  continues  with  its  own  execution 
until  it  needs  the  AP  again.  At  this  point  it  waits  on  the  completion  of  any 
previous  Al’  call  by  looping  v’J.th  an  EXF  to  test  AP  activity  (all  SARREX 
routines  end  with  a WAIT  instruction).  PARPJiiX  then  reloads  /iPARG  and  the 
cycle  repeats.  Within  SARREX  the  AP  argument  area  is  known  as  SAPARG. 

The  main  SP  (sequential  processor)  module  is  named  APL.  It  is  written  in 
Macro-11  (assembler  language  for  the  PDF-11)  and  is  linked  with  a number  of 
subroutine  modules  (also  written  in  Macro-11)  by  the  LINl'-ll  system  program. 
Wlien  loaded  it  occupies  PDP-11  octal  addresses  from  about  30000  through  137000. 
The  addresses  from  100000  through  137000  are  shared  with  tlic  Associative  Pro- 
cessor starting  at  AP  control  memory  address  BOOO  (hexadecimal).  Tliese 
addresses  contain  tb.e  communications  areas  such  as  AP/.RG.  The  othe.r  modules 
linked  with  APL  are  named  PARREX,  FARKiLX,  SUPER,  INSTAR,  EXECFl’,  and  DUMP. 

A number  of  subroutines  from  the  Fortran  System  Library  are  also  called  by 
routines  in  FARREX  and  are  incorporated  into  the  load  module  when  it  Is  bui.lU 
by  LINK-11.  The  I'DP-ll  transfer  address  is  named  START  and  is  located  at  r.-’i.o- 
catable.  0 in  the  AP).  module.  Tliis  module  is  the  master  control  for  tlie  entire 
APL-STAR/)N  System. 
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PARREX  has  10  entry  points  named  ARREX,  LDATA,  SDATA,  OUTSIG,  FIX, 
FRAFIX,  FFLOAT,  NUMFUZ,  NUMCON,  and  $ERRA.  All  entries  to  PARREX  coma  from 
APL  as  subroutine  calls.  When  an  error  condition  In  the  AP  forces  the  dis- 
continuance of  a PARREX  routine,  the  return  to  APL  uses  the  ERROR  entry  In 
the  APL  module  Instead  of  the  normal  subroutine  return.  The  various  func- 
tions provided  by  the  10  cntiT  points  to  PARREX  are  described  in  the  docunen- 
tatioii  contained  in  the  PARREX  source  code  listing. 

Tlie  PARREX  module  Is  a collection  of  9 Fortran  subroutines  used  by 
PARREX  to  perform  ceri'ain  sequential  operations,  such  as  the  sum  and  product 
reductions.  There  arc  also  a few  array  oriented  opcratloirs  namely,  logarithm, 
exponential,  floor  and  celling,  which  are  not  Implemented  as  array  operations 
in  version  1.1  of .APL-STAKAN.  In  these  cases  PARREX  uses  SARREX  to  bring  the 
data  over  to  the  PDP-11  where  the  calculations  arc  performed  serially  by  rou- 
tines in  the  FARREX  module. 

SUPER  is  used  by  the  APL  module  to  handle  the  interface  v;lth  the 
DOS-11  BATCH  operating  system.  It  h^s  one  entry  point,  SUPER,  and  the  spe- 
cific function  requested  is  determined  by  a control  code  which  must  have  been 
previously  stored  by  the  caller  at  SCNTRL.  The  entry  is  of  the  form  of  a 
subroutine  call  but  the  normal  return  sequence  will  be  replaced  by  a jump  to 
ERROR  (in  APL)  if  an  abnormal  condition  occurs.  Documentation  found  in  the 
source  code  listing  describe  the  specific  functions  provided  by  SUPER. 

EXECFN  is  the  APL-STARAl^  compiler,  which  has  been  written  in  APL- 
STARAN  and  then  hand-translated  into  QCODE.  Tlie  EXECFN  module  is  the  QCODE 
copy  of  this  compiler  . A source  listing  in  APL  may  be  found  in  the  appendix 
to  thlfi  report.  In  this  source  listing  the  functions  QESCOO,  QESC02,  QESM02, 
QESM04,  QESM06,  QAQESC,  and  QAQESC4  are  qcode  escape  functions  which  are  not 
available  as  ordinary  APL-STARAN  functions,  but  are  used  by  the  compiler  to 
perform  such  tasks  as  sjnnbol  table  mr-jilpulation,  etc.  The  precise  details 
of  how  these  functions  operate  may  be  found  in  the  source  code  of  the  main 
APL  module. 

The  DUET  module  is  residual  debugging  instrumentation.  Its  entry 
point  is  at  absolute  octal  address  130000,  but  there  arc  no  active  references 
to  that  address.  When  the  dump  function  is  required  it  is  necessary  to 
patch  the  ERROR  routine  In  the  APL  module  to  effect  a jump  to  the  dump  entry 
point.  Tills  may  easily  be  accomplished  by  replacing  thp  DO.S  coiiunand  $RUN 
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I)K1:M’L  with  a $GET,  $MODIFy,  $BEGIN  sequence.  The  effects  of  a call  to 
DUMP  will  be  to  create  a Multlcs  file  APL.DMP  which  will  contain  most  of  the 
data  fn  the  APL-STAEAN  system  tables  and  communications  areas.  APL-STARAN 
will  then  be  terminated,  the  AP  will  be. cleaned  up,  and  control  will  be 
returned  to  the  DOS-11  Operating  System  (unless,  of  course,  the  bug  has 
destroyed  the  dump  routine) . 
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APPENDIX:  THE  APl.-STAKAN  COMPILER  (IN  APL) 


1 


V VARES^EX  VATC 
[ 1 ] V/  RZS-*-VA yN*-CtEMPT 

[2]  VACP<-~l  + VAOR-*-IB  48 

[3]  VAIC-^^SIVAIC 

[4]  LASKIPi^LASKIP  IF  Z2-VAIClVACP’*-VACP*l'i 

[5]  -^LAGO  IF  6B*VAICLVACP'] 

[6]  -^-LAGO  IF  Ves'V'/lICCK/lCP+l] 

[7]  VAIC\.VAQ*VAOR*~l*f)VAIC'i*-\B 

[8]  ^LAFERD 

[9]  LAFIN:VAQ->-VAORi-'‘l^pVAIC-*-VAIC,(9SlOT  #I),13 

[10]  LAFEtWi*LAFEVD  IF  Z7-VAIClVAQ-*-VAQ-\l 

[11]  ^LAFIN  IF  76>^VAICLVAQ'] 

[12]  ^LAFIR  IF  60*VAICLVAQ-11 

[13]  VAIClVAQ^Xl-^B 

[14]  VAFN-0 

[15]  LAGOiVACP-*-VACP-l 

[16]  LATRAN:-*-LAPUT  IF  8<VAQ-*-CtPT0QiVA0R*VAICZVACP->-VACP^l'i'l 
[17  ] LACO^LACl  ,LAC1  ,LAC3  ^LACH  ^LATRAN  ^LACB  ^LACl  ,I.ACG)LVAOR*VAQ 

] 

[18]  LAC3:-*-LACl  IF  l^CAPTOQlVAOR+VAIClVACP*!}'} 

[19]  l^A(2-^14 

[20]  LAPUT:VARES-^-VARES,VAQ 

[21]  -*-LATRAN  IF  17  *VAQ 

[22]  QESHO&  1 

[23]  ^LATRAN 

[24]  LACO:-*-LAPUT  IF  VAZ^6<VAQ->-CAMT0QZCtPT0M\VAIClVAW-^VACP*VAGP*l 
'}  + 250y^VAICiVACP21 

[25]  -^LACOCO  IF  5>VAQ 

[26]  h-L/ICG  if  8 = V^/I« 

[27]  LACG-.QAQESCi* 

[28]  LACOLP:VAZ*-{2tB>^VAZ)-^VAX’SH-ti2*^VAI 

[29]  LACOGOf*-I.ACOLP  IF  liVAy->-CtPT0CdVA0Ri^VAX*VAICZVACP->-VACP+112 

[30]  ^LAC6  IF  "dyVAy-VACP-^VACP-l 
[3n  ^LACQSM  IF  VAQ*H 

[32]  VARES^VAREG  .(1^/42  + 208).  VAQ 

[33]  ^LATRAF 

I [34]  LACQSM:->LACB  IF  VAV^VACP 

h [35]  ^LACOIN  IF{VAY<-pVALSYM)>VAX^{VAtiSIM\VAZ<-VAZ*VAQ>fB>i072)- 

■ VAOR 

1[  3 6 ] VA  AS17/,  ( ( ( VAX*QESMO  2 CATYPEl  VA0R-tVAQ2) -VAY)pO)  ,VAZ 

[37]  LA  COIN : VA  RES^  VA  RES , ( CLSMSTRT-  1 6 >*.  V/i  X)  ^VAQ 

[3  8]  -^LATRAR 

[39]  Z,y!Cl  : V/lQ-^-CAPA/PP 

[40]  L/ClffO:  l 

[41]  F/lE-^54872 

f [42]  VAX^Q 
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[43]  LAClLPi-^'iLAClAL  ,LAC1B0 ^LACISC ^LACIDV ^LACIGNKVAOR* 

4 L CAPTOQ  IVAOR^VAW^VAICZ  VA  CP-»-VA  CPf  1 ] ] ] 

[44]  LAC\NO:VAX-*-{10xVAX)i-VAW~ne 

[45]  -^LACILP 

[46]  LAClSGi-*-LAClLP  IP  VAV=VACP^VAI*-‘~1 

[47]  -^-LACB 

[48]  LAC\nPi-*LAClLP  IF  VACP=VAZ*-VACP^VAZ-^i^B12 
[4  9]  -^LAC^ 

[50]  LAC\ALi-*LAClGR  IF  69*K/IM 

[51]  -^LACINE  IF  9H  = VAP-<-VAIClVACP+VAW-*-ll 

[52]  -^iLAClGN  ,LAC1EX)IVA0R^1=CAPT0QIVA0R*VAP'}1 

[53]  LAClNEx-*LAC&  IF  l*CtPTOQlVAOR*VAP*-VAICLH^VACP->^VACP-VAW-*- 

"1]] 

[54]  VAZ*-VAZ  + 1 

[55]  LAC1EX:VAP^VAP-HQ 

[56]  -*-Li4C6  IF  i^6  = VAR*-VAICiVACF^VACP*2l 

[57]  ^LACIGS  IF  IrCLPTOQlVAOR+VARl 

[58]  VAZ^-VAZ  + l 

[59]  l^/lP-H(10xl'4P)  + 7i4/?-48 

[60]  -^LACG  IF(3  = VAR)vl  = VAR*C&PTOQCVAOR+VAIC[yACP^VACPi-llJ 

[61]  LAClGE:VAP^10*'VAf/xyAP 

[62]  VAZ^VAZ\2 

[63]  LAC\GE  v->-LAC^  IFC\-yAX^VAI')>i'‘3-yAV\VAZ-2y-VACP 

[64]  VAQ-*-VAQ,VAYyVAPy^VAX*10*0lVAZ~VACP^VACP-l 

[65]  LACINX  LA-CIND  ^LAClGO  ,LAC1G0  ,LAC1G0  ,LAC6  ,LAC1NX  tLACe  , 

LAClND)iVAOR+7lCliPTOQLVAOR+VAICLVACP-*-VACP*l2'i'l 

[66]  LAC\IJD'.’>-LAClEtl  IP  l*f>VAQ 

[67]  VAQ^CLEMPToVAQ 

[68]  LAC1EN'.VARES^VARES^{VAFR  QESCQQ  7>IQ),5 

[69]  VACP*-VACP-1 
r70]  ^LATRAR 

[71]  LAChiVAQ-^CLEMPT 

[72]  LACULPi-^-LACHQT  IF  39  = VAX*-VAICiVACP^VACP-H'i 

[73]  ->-LAC6  IF  3 = VAX 

[74]  LAC^GC:VAQ-*-VAQ»VAX 

[75]  ->L/1C’4LP 

[76]  LACHQT:-*-LACUGC  IF  39  = VAIC^VACP‘*^VACPi■ll 

[77]  VAn*VAQ,CLEMPT 

[78]  -^LACIND 

[79]  LACOz-^LAPUT  IF  17  ^VAQ^CEPTOQZVAOR+VAICLVACP^VACP*!!! 

[80]  -^-LACB  IF  7*VAQ 

[ 81  ] LAC7Z  VACP-^VAOR  + pVAIC^e  , 7 , VARES 

[82]  VAA^-VAB^VAL-i-VARES-^CEEMPT 

[83]  VAV*-Q^VAZ^V.AOR 

[84]  VAW->rVAX<-VAY->‘3pVAJ*-Q 

[85]  -^L/iSLP  IF  0 = pVAFH 

[86]  VAP<-VAA*-VAB^Q 

[87]  ^LACb  IF{pVAIC)  = {VACP*^VAIC\n)-VAOR 
[80]  VAICiVACP']*-!^ 

[89]  QESMOb  0 
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[90]  LALOCLt->-LALCND  IF  2(i*VAICiVACP*VACP-Z'\ 

[91]  -*-AylC6  IF  0*VAICIVACV*2Z 

[92]  VAL*-VAL,VAICiVACP^\'S 

[93]  -^LALOCL 

[9*t]  LALClWi^LAFh'AM  IF  0*VAIClVACP^2'\ 

[95]  VAB<-VAICl3*VACP-*-VACP-2l 

[96]  LAFNAM:-^LAC&  IF{0  = VAQ)yf3<VAQ^VAICZVACP*2l 

[97]  VAFN^,VAIC[VACP*l'i 

[98]  -^LANOLA  IF  0*VAIClVACP'i 

[99]  VAA^VAIC{\^VACP^VACP’-2'i 
{.IQOlLANOlAi-^-LANORS  IF  21*VAIClVACPl 

[101]  -^L/lC6  IF  0:'-VAICZVACP”ll 

[102]  VAP<-VAIC\.\iVACP-*-VACP~2l 

iXQZZLAlWRS'.^LAC^  IF{VAQ*l*{^VAA)^*VAB)^{VACPxVf.OR*2)v2Z.*VAICi 
VACPZ 

[104]  VARES*-XZ^^VAA^VAU^VAP,VAh^O 
[10  5]  VAA.-*-VA  B*-V  f ''  AL*- » 20  8 

[106]  K/.<7-*-l 

[107]  Q^5W06  1 

[108]  -*-LASLP  IF{VAOR*pVAIC)*VACP*-VAIC\17 

[109]  ->-LAC6  IF  28*VAIClVACP*-yACP~l2 

[110]  ^LAFOV-T  IF  18-VAICiVACF~l'i 

[111]  V/1JC-^1';1JC,17 .28  ' 

1112'iLASLPi  VAR^(iVARES*-VARES  ,{{~VAJ)plZl)  ! 

[113]  -^LAC^  IF  VAJ*-iVAP=^0)^{VAP*2b)h8<VAP^CUQTOClVAORiVAICiVACP--  I 

I'^JCP-l]] 

il\^']IASYNiVAQ^-VAP  | 

il\b'\LAFEXTi->'LAKEXT  IF  208%VAP-*-VAICS.yACP-^VACP-l'\  \ 

[116]  VAP-<rCLQTOCiVAOR^VAP'\  \ 

[117]  -^iLAA  ,LAn  ^LAC  ^LAD  ,LAE  ,LAD  ,EAG  ^LAD  ,LAI  ,LAJ  ,LAK  ,LAL  ,LfiM  ,LAH  ^ f 

LA0,LAP,LAQ,LAR,LAS,LAT,LAl/,LAV^LAV,LAX,IAr»LAZ,LAC6,LAC6)  ( 

iVAOR^VAQl  I 

CJ1B]LAA:-^LAA0P  IF  15  = VAIC[VACPJ  I 

[119]  ->-LACb  IF(ll*VAP)/^13rVAP  | 

[120]  VARES^VARES  ^lH^VAICCVACPl  i 

[121]  VAICLVACPl-^-20  i 

[ 1 2 2 ] L/. >4 or ; I'/!  P<-CLQTOC [ VAORi  VAIC [ VA CP-^ VA  CP~  1 ] ] | 

, [123]  ^LAf4  I 

if,  [12!i]L/.P:l'/lPPS-‘T/.PP5,  12  24  204  132  ji 

" [125]  ^LADU  f 

\.l28'llACxVARES‘>rVARES  ,1^  \ 

[12  7]  LA  D : VA  RER*-  VA  RES  , IM  I(?  [ U/.  CP<  1 ] 1 

112  ?']LAP!i -.->1X311,’  iy(6-^VAr)t,2HiVAP  | 

[129]  -^LACe  f 

[i3r.]L/:P:  i'yiy[iViZ]H.v/ir[iMZ]  + i f 

[131]  LAE : IMPP.5'-  VA  RES  ,VAIC{.  VA  CPi  \ ] | 

[132] L.4P.'.<:-:£/5i7;  iT(  1 Sl'/P) A6i ^'/lP  | 

[133]  ■*LAC8 

[ 1 3 u ] /./  0 : -^LA  CG  IF  VAV<  VA  Z-  VA  7.  + 1 
1 [121']  VAVLVA:’.]*-VAaI  VA  Z J VA  0 R ^pVA  RES 


[136]  -^LAGEX  JF(lsr-4P)A6iK^P  ’ 

[137]  ■*LAC&  ir{l^*^VAP)h70*VAF 
[136]  VARES^VAHES 

[139]  ^LASXN 

{.\u,qZLAGEX  ‘.VARIS^VARES  AO 
[I**!]  -^LASYR 

[142] L>lI:-*-L/lP  IF  0<.VAYiVAZl*-VAYiVAZl-l 

[143]  -t-LACe 

llUH  iLAJ  : VAREG^VARES  AO 

[14  5]  ->L/1P 

[146] L-4;f:  I'XlP£'5-^K>iP£'S,28,KviIC[7>lC'P+l],29 

[147]  -^-LAEM 

[ltt8UAL:VA RES->- VARE5,  VA IC [ VACP^  1 ] 

[149]  -^LASYN  IF  OiiVAP 

[150]  -^LACO 

ll51']LAQi-*LAC0  JP(  34  = Ki4P£5[  IMOP+'H-p  IMPP5]  ) a6  < l^/IP 

[152]  ->^T,ARQ  TF  VAQv&<VAP 

[153]  --LARQUI 

[154] L/lP:->-L/ll.'':  IF  12Z  = VARESiVA0R-\''X^QVARES'i 
[ 1 5 5 ] L.4  RQHI : V/  h ES-*-VA RES  ,135 
[156]LylP<3:-*-L/c:6  IF  1'/1P  = 0 

[ 1 5 7 ] L>5  A' : y/1  PFS  , I'yl IC[  1^4  (7P+ i ] + 1 2 8 X 6 < 74 P 

[15C]  -^LASYN  IF  24 2 74P 
[159]  -•■L4C6 

iieolLAOi-^-LAK  IF  O^VAP 
[161)  -»-L46’6 

[ 1 G 2 ] L4P : 74P2?S+-74PZi’5 , 74IC[  74 CP-i  1 ] 

[163]  -t^LASYR  Jf(5  = 74P)v6  = 74P 
•[164]  -►L4C6 

Cl6  5]li4S;-*-L4.9T'.9r  IF  133  = 7APP5[740Pt“l+p  74PP5] 

[166]  74PPS-«-74;.’P5M35 
llbT]LASTST:-*LAM  IF  (lirVAP)  M3*VAP 
[16  8]  FARES*-  VA  RES . 74  JP  [ VA  CP-i  1 ] - 6 2 

[169]  VAP^-CRQTOCLVAORirVAIClVACP^-VACP-lll 

[170]  -^LAD 

InilLATi^LACO  IF  74  P/C  74  Z ]'^0  x 74  y[  74  Z ] 

[172]  *-LATEX  JF(1:574P)a6274P 

[173]  h-PAC’G  JP(  19x74P)A20x74P 
[17  4]  74  RES*-  VA  RES  ,25,24 

[17  5]  *-LASYN 

[ 1 7 L ] L4 TEX  ; 74PPS- 74 PIT" , 2 6 
[17  7]  -^LASYN 

[ 1 7 8 ] Z,4  P : ->-/.4  C 6 7P  ( 74  6>P > 74  Z ) v 0 74  T [ 74  Z ] 

[179]  -*LAUnS  JF(74P=17)v74P=10 

[100]  *-LAUNFN  IF  7/;P?:lG 

[ 1 0 i ] 74 1 C [ 74  P P-i- 1 ] ^ 74  7 C [ 74  PP + 1 ] -H 

[ 1 8 2 ] L4  URS ; 74  RES*-  VA  RES , 74  IC  [ 74  6’P+ 1 ] + 1 1 1 

[103]  -74 C6  7P(74f<'[74Z]  = 0)v74/?P5[74y[74Z]]-~S 

[18  4 ] 74  RES  [ 7/1  y [ 74  Z ] ] - 8 

[10  5]  -yLAUOK  IF^18  = CAQTOC^VAOR■^700^VA  RES  [ 74  Q - 1 ] ] ) v 1 2 8 5 74  RES  [ 74  Q*- 
74A  [ 74Z]-1] 
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[186]  -^LAUOK  IF  26=CtQT0CLVA0nf7.0QlVAnESiVAQl-*-VARESlVAQ’]* 

120] 

[187]  ^LAC& 

11B82LAUNFN:-*LAUZ  IF  2S  = VAP 
[109]  ■t’LACt  JF(0  = V/5P)v5<1'/1P 

[19  0]  VAFFS^-  VA  FES  ,VAIC[VACP&l'ifVAQ->-21  = VAFESZVAXlVAZl-ll 
[191]  -^-LA  CG  IF  (IB  =CAQTOCZ  VA  GIU2  08  L VARESl  IM  1 ] ] ) v(  12BiVARESZ  VAQ-^ 
VAXL  K/l  Z ] - 1 ] ) V C A K/lPje  6 
Zl82']LAUOK:VAZ-^VAZ-l 
[193]  ^LASYN 

Zl9^'iLAUZ:->-LACG  JP(  74  VC  K/ Z ] = 0 ) v(  74Pff5[  74  Ar[  74  Z ] ] = 9 ) v74PE5[  74;r[  74  Z 
]-l]»=lV 

[195]  74PP5[74A'[74Z]-1 ,0]'^16.17 

[196]  VAQ^pVARES*-VARES , 130  0 

[197  iLAUZLPtVAD^VAB,  p VARES^VARES  . 74IC[  1 + 74CPk-74CP-2  ] 

[198]  -^LAUZLP  IF  2 5 =CA02'OC’[  740P+ 74  JC[  74(7P]  ] 

[199]  VARESlVAOR  + VAQ-l']*-(pVARES)-VAQ 

[200]  74Z-*-74Z-l 

[201]  ■*hAZGO 

[202] L47:H'r.4C6  IF  28*VAP 

[203]  VA.RES^VARLS  ^\29 

[20*4]  74  74  CP -2 

[205]  -^LAZPUT 

[2  06]Z;4r/:-»L4C6  IP(  1 6 >'74rC[  74 CP- 2 ])  v(  2 5»:74P)  v(  74CPt  p 74£ ) ?'74i  i 74Q-^ 
74JC[74CP-1] 

[207]  VAL-^VAL^VAQ 

[208]  744-^744, 74P 

[209] L4X:-»-Zi4CG  JP(  0^p74P//)  v(  74Z-74Cx7)v74y[  74CP]s‘0 

[210]  QESM 06  1 

[211]  -^LANOLD  IF  74  Jv(  21  = 74P)  v ( 2 3 = 74P)  v ( 7 8 - 74P)  v ( 79 -74P-^74P£:£[ 
74Q-l])v(  206  = 74P)v207  = 74P-^74PC5[74C-‘-740P+"l  + p74P£:S] 

[212]  74PPS^74PES,139 

[ 2 1 3 ] L4  A’ CLD  : 74 IC [ 74  JC 1 1 7 ]-H  6 

[214]  -^LASLP  IF(VACR-{oVAIC)*VACP*-VAIC\n 

[215]  -*LAC6  IF  2 8»-74JC[74CP'<-74CP-l  ] 

[216]  VAIC->-VAIC  ,n  ,28 

[217]  -^LASLP  IF  16/74JCr74CP-l] 

[216]  74 4 -^  ( 1 + p 74  PCS ^ 7/i  PC5 ,( ~ 74  J ) p 1 3 1 ),  74 4 

[219]  ^LAFOUT  IF  0 = p 74P 

[220]  74  V*  VA  OP  + p 74  JC^  74  0P+  74  fl  - 1 

[221]  742-740P  + p74P 

r 2 2 2 ] i4  i EUR  t-^-LACb  IF  74  Q = VA  P'‘'74 L \ VARES [ 74  CP-^ 74 IC[  74  Z ] ] 

[ 223  ] 74  P P S LI  '4  C 2 ’]+•  2 X 74  /.  L 74  P ] - 74  P [ 74  Z ] 

[224]  -yLAINBR  IF  74  7 > 74  Z-^74  Z + 1 
[2?5]P4PO(77’:-'74/7/  QESC02  VARES 
[22  6 ]2:,4r  :-;L4CG  IP ( 74  Zj^ 74 OP ) v 74  Y[  74 OP ]:^0 
[ 227  ] -^QESMOh  VARES 

[ 228  ]L4Z  : 74PPC[  740P  + "l  + p 74Pi’P  ]-^  1 28 
[ 7 2 9 ] P4  Z 'WT : VAB--VA  li , p 74  PPS+  74  RES , 74  JC[  1 -i  74  CP] 

[ 230  ]P4;  (;0:->-RASyN  IF  VAJ*-(  ( 2 2=^VAP) 3=VAP  ■ CLQTOCZVA  ORiVAICZVACP-t- 
74  CZ'- 1 ]])  A 1 2 8 --  74  JC [ 74  CP] 

[2311  -*L4C6  V 
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